Commit e5ad0886 authored by Duncan Deveaux's avatar Duncan Deveaux
Browse files

Improved usability of the scripts

parent 22e5414d
......@@ -2,38 +2,40 @@
Estimation of the relationship between the variation of TTC in an area and the amount of associated risk.
# How to use
# Files description
* To use the provided code, `ROUND_PATH = "path/to/round/data/"` should be replaced with the path to your local copy of the rounD dataset in the [topology.py](topology.py) file.
The provided TTC analysis code is a tool to assess the variation of TTC in roundabouts of the rounD dataset, as well as risk, and investigate on correlation between these two units.
The provided files should be run in the following order:
* [visualization.py](visualization.py): Provide tools to visualize the tracks of all vehicles from a given tracks file. As well as a tool to replay the tracks frame by frame.
* [parse.py](parse.py): Parses the provided tracks files (all from a single Location ID) to augment the tracks with TTC data.
* [parse_exits.py](parse_exits.py): Parses the recordings of a given location ID to extract training data to train a model to assess roundabout exit probability of vehicles.
* [parse.py](parse.py): Parses the recordings of a given location ID to augment the tracks with TTC data.
Once the data has been generated, it can be analyzed and visualized with the following scripts:
* [analysis_exits.py](analysis.py): Trains a model to assess exit probability of a vehicle from the roundabout of given location ID.
* [analysis.py](analysis.py): Uses the TTC-augmented tracks generated by the [parse.py](parse.py) file to study correlation between TTC variation and risk.
# How to add locations
For the moment, support for the location of ID 0 is provided.
In order to add new locations to the study, the following process should be followed:
1. A Topology description has to be generated for the wanted location ID. The Topology description is a simple class instance which indicates the center point of the roundabout. It also provides a list of circular lanes that the roundabout contains.
1. In order to define the Topology object, please refer to the comments and tools in the [visualization.py](visualization.py) file.
* Replace the `input_file`, `input_meta_file` and `input_recordingMeta_file` with a tracks file of the wanted location ID.
* A plot of all tracks of the file will appear. Use it to choose the center point of the roundabout and the distance of lanes to the center of the roundabouts.
* Each lane should be 2.25meters long, see [topology.py](topology.py) for a simple example of a topology for Location 0.
* Play with the `topology` object in line 85 of [visualization.py](visualization.py) until the generated blue lanes accurately wrap the tracks.
<br/>
2. Use the [parse.py](parse.py) file to generate TTC-augmented data for the wanted location.
1. Replace the `input_ids` variable with a list of all tracks id of the wanted location.
2. **Do replace the `topology` object on line 73 with the new topology you generated in the first step.**
3. Upon running the updated script, TTC-augmented data will be generated in a newly created ttc_parse/ directory.
<br/>
3. Use the [analysis.py](analysis.py) file to generate the scatter plots and correlation analysis between the variation of TTC values and the amount of risk.
1. Replace the `input_ids` variable with the list of all tracks id of the wanted location.
2. Run the script to see the plotted results!
# Quickstart
```console
python3 parse_exits.py --location=[INSERT YOUR LOCATION HERE]
# This will generate the training data for roundabout exit probability model training.
# Pickle files will be generated in the exit_parse/ directory.
python3 parse.py --probability_weighting --location=[INSERT YOUR LOCATION HERE]
# The probability_weighting option is important, as it will allow the parse.py script to use the exit probability model to generate probability-weighted risk.
# Pickle files will be generated in the ttc_parse/ directory.
python3 analysis_exits.py --location=[INSERT YOUR LOCATION HERE]
# Will show visualization of the training data for the roundabout exit probability model.
# Will train the model and give its accuracy.
python3 analysis.py --location=[INSERT YOUR LOCATION HERE] --ttclimit=7.5 --cvtime=60.0
# Will show correlation analysis between TTC CV and risk.
# --ttclimit is the maximum TTC value in seconds that will be considered for computing the CV.
# --cvtime is time in terms of TTC data values that will be used to compute the CV. (e.g. compute the CV of TTC values over the last 60 seconds of TTC values).
```
......@@ -16,10 +16,12 @@ import pandas as pd
import seaborn as sn
from scipy import stats
import pickle
import argparse
import roundtools.read_csv as rd
from topology import ROUND_PATH, Topology, Lane
from ttc_correlation import TTCTimeline, TTCData, VariationDataset
import locations
def plot_pattern(ttcdata, risk_mode):
......@@ -48,24 +50,28 @@ def plot_pattern(ttcdata, risk_mode):
# Plot correlation data from the pickle files generated by parse.py #
# ................................................................. #
# TODO: Provide here the list of tracks ID for the desired LOCATION
# Note: These are the ids for location ID=0 in my case.
input_ids = ['02','03','04','05','06','07','08','09']
for i in range(10,24):
input_ids.append('{}'.format(i))
parser = argparse.ArgumentParser()
parser.add_argument("--location", help="The location ID to analyze.", type=int)
parser.add_argument("--input", help="The directory where the pickle files generated by parse.py are located.", default='ttc_parse')
parser.add_argument("--ttclimit", help="The maximal TTC value to be considered to compute TTC values variation, in seconds.", type=float, default=7.5)
parser.add_argument("--cvtime", help="The amount of TTC data history to be considered when computing the Coefficient of Variation, in seconds.", type=float, default=450.0)
argsparse = parser.parse_args()
input_ids = locations.get_input_for_location(argsparse.location)
print ("Input files for location {}: {}".format(argsparse.location, input_ids))
ttc_data = []
for id_str in input_ids:
with open('ttc_parse/round_ttc_{}.pickle'.format(id_str), 'rb') as f:
with open('{}/round_ttc_{}.pickle'.format(argsparse.input, id_str), 'rb') as f:
result = pickle.load(f)
# Break the TTCTimeline into 5 minutes sections
framerate = result['timeline'].framerate
averaging_step = 1.0
time_break = 450.0 # in seconds
time_break = argsparse.cvtime # in seconds
sub_ttcdata = TTCData.from_ttc_timeline(result['timeline'], 7.5, averaging_step, time_break*framerate/averaging_step)
sub_ttcdata = TTCData.from_ttc_timeline(result['timeline'], argsparse.ttclimit, averaging_step, time_break*framerate/averaging_step)
ttc_data.extend(sub_ttcdata)
......
......@@ -12,9 +12,10 @@ plt.rcParams['ps.fonttype'] = 42
from sklearn.linear_model import LogisticRegression
import numpy as np
import pickle
import argparse
import roundtools.read_csv as rd
import locations
from exit_tracking import ExitTracking
# ................................................................. #
......@@ -97,11 +98,12 @@ Run script to generate graphs about the trained model
'''
if __name__ == '__main__':
# TODO: Provide here the list of tracks ID for the desired LOCATION
# Note: These are the ids for location ID=0 in my case.
input_ids = ['02','03','04','05','06','07','08','09']
for i in range(10,24):
input_ids.append('{}'.format(i))
parser = argparse.ArgumentParser()
parser.add_argument("--location", help="The location ID to analyze.", type=int)
argsparse = parser.parse_args()
input_ids = locations.get_input_for_location(argsparse.location)
print ("Input files for location {}: {}".format(argsparse.location, input_ids))
training_data = gather_training_data(input_ids, 101010)
......
#!/usr/bin/env python3
'''
Author: Duncan Deveaux
'''
from os import listdir
from os.path import isfile, join
import roundtools.read_csv as rd
import topology
def get_input_for_location(location):
input_ids = []
data_files = [f for f in listdir(topology.ROUND_PATH) if isfile(join(topology.ROUND_PATH, f))]
for filename in data_files:
if filename.endswith('recordingMeta.csv'):
meta_info = rd.read_meta_info({'input_meta_path': join(topology.ROUND_PATH, filename)})
if int(meta_info[rd.LOCATION_ID]) == location:
split = filename.split('_')
if len(split) == 0:
print("Warning: filename {} is not separated by '_'".format(filename))
else:
input_ids.append(split[0])
return input_ids
def get_topology_for_location(location):
if location == 0:
return topology.Topology.roundDLocation0Topology()
elif location == 1:
return topology.Topology.roundDLocation1Topology()
elif location == 2:
return topology.Topology.roundDLocation2Topology()
else:
raise Exception("The topology for location {} has not been defined.".format(location))
......@@ -24,6 +24,7 @@ from topology import ROUND_PATH, Topology, Lane
from ttc_correlation import TTCTimeline, TTCData, VariationDataset
import analysis_exits
import locations
def get_frames(tracks_meta, trackId):
......@@ -56,11 +57,16 @@ def read_frames(last_frame=-1, last_track=-1):
# analysis.py .............................................................. #
# .......................................................................... #
# TODO: Provide here the list of tracks ID for the desired LOCATION
# Note: These are the ids for location ID=0 in my case.
input_ids = ['02','03','04','05','06','07','08','09']
for i in range(10,24):
input_ids.append('{}'.format(i))
parser = argparse.ArgumentParser()
parser.add_argument("--dst", help="Directory where the generated pickle files will be saved.", default='ttc_parse')
parser.add_argument("--location", help="The location ID to generate TTC data from.", type=int)
parser.add_argument("--probability_weighting", help="Compute risk using vehicles' roundabout exiting probability.", action="store_true")
argsparse = parser.parse_args()
print ("Loading recordings for location {}".format(argsparse.location))
input_ids = locations.get_input_for_location(argsparse.location)
print ("Input files for location {}: {}".format(argsparse.location, input_ids))
input_args = []
for input_id in input_ids:
......@@ -69,24 +75,15 @@ for input_id in input_ids:
"input_meta_path": ROUND_PATH+input_id+"_recordingMeta.csv" })
# !!!!!!!!!!
# !! TODO: Provide the topology associated with the wanted Location
# Here is the topology for location ID=0 as an example.
# See visualization.py to define the topology of new roundabouts.
topology = Topology.roundDLocation0Topology()
# END TODO
# !!!!!!!!!!
# Loading topology for the location.
topology = locations.get_topology_for_location(argsparse.location)
# .................................................................................... #
# If requested, compute exiting probability estimation models to weight TTC risk with. #
# .................................................................................... #
parser = argparse.ArgumentParser()
parser.add_argument("--probability_weighting", help="Compute risk using vehicles' roundabout exiting probability.", action="store_true")
args = parser.parse_args()
exit_proba_models = {}
if args.probability_weighting:
if argsparse.probability_weighting:
print("Computing exit probability models...")
for input_str in input_ids:
......@@ -102,7 +99,7 @@ if args.probability_weighting:
# Create the target directory if needed
try:
os.mkdir('ttc_parse')
os.mkdir(argsparse.dst)
except FileExistsError:
pass
......@@ -146,6 +143,6 @@ for args in input_args:
timeline.add(frameId, obj[rd.TRACK_ID], ttc[0][rd.TRACK_ID], ttc[1], risk_probability)
result = {'timeline': timeline}
with open('ttc_parse/round_ttc_{}.pickle'.format(args['id']), 'wb') as handle:
with open('{}/round_ttc_{}.pickle'.format(argsparse.dst, args['id']), 'wb') as handle:
pickle.dump(result, handle)
......@@ -17,10 +17,12 @@ import pandas as pd
import seaborn as sn
from scipy import stats
import pickle
import argparse
import roundtools.read_csv as rd
from topology import ROUND_PATH, Topology, Lane
from exit_tracking import ExitTracking
import locations
def get_frames(tracks_meta, trackId):
......@@ -53,11 +55,13 @@ def read_frames(last_frame=-1, last_track=-1):
# analysis.py .............................................................. #
# .......................................................................... #
# TODO: Provide here the list of tracks ID for the desired LOCATION
# Note: These are the ids for location ID=0 in my case.
input_ids = ['02','03','04','05','06','07','08','09']
for i in range(10,24):
input_ids.append('{}'.format(i))
parser = argparse.ArgumentParser()
parser.add_argument("--location", help="The location ID to generate TTC data from.", type=int)
argsparse = parser.parse_args()
print ("Loading recordings for location {}".format(argsparse.location))
input_ids = locations.get_input_for_location(argsparse.location)
print ("Input files for location {}: {}".format(argsparse.location, input_ids))
input_args = []
for input_id in input_ids:
......@@ -65,15 +69,8 @@ for input_id in input_ids:
"input_static_path": ROUND_PATH+input_id+"_tracksMeta.csv",
"input_meta_path": ROUND_PATH+input_id+"_recordingMeta.csv" })
# !!!!!!!!!!
# !! TODO: Provide the topology associated with the wanted Location
# Here is the topology for location ID=0 as an example.
# See visualization.py to define the topology of new roundabouts.
topology = Topology.roundDLocation0Topology()
# END TODO
# !!!!!!!!!!
# Loading topology for the location.
topology = locations.get_topology_for_location(argsparse.location)
# Create the target directory if needed
......
......@@ -152,16 +152,32 @@ class Topology:
exit_points = [(97.2,-24.3), (55.5,-35.8), (65.3,-70.0), (106.0,-59.0)]
print("Generated Location 0 Topology...")
return Topology(roundabout_center, circular_lanes, exit_points)
# TODO! Define the lanes position and roundabout center for other topologies : the following pattern can be used:
'''@staticmethod
def roundDLocationXTopology():
roundabout_center = (-1, -1)
circular_lanes = []
@staticmethod
def roundDLocation1Topology():
roundabout_center = (115.6, -70)
circular_lanes = [Lane(roundabout_center, radius_begin=8, radius_end=10.25),
Lane(roundabout_center, radius_begin=10.25, radius_end=12.5),
Lane(roundabout_center, radius_begin=12.5, radius_end=14.75)]
exit_points = [(121.0, -52.0), (97.8, -68.5), (133.2, -75.0), (111.0, -89.0)]
return Topology(roundabout_center, circular_lanes)'''
print("Generated Location 1 Topology...")
return Topology(roundabout_center, circular_lanes, exit_points)
@staticmethod
def roundDLocation2Topology():
roundabout_center = (138.0, -61.3)
circular_lanes = [Lane(roundabout_center, radius_begin=6.75, radius_end=9),
Lane(roundabout_center, radius_begin=9.00, radius_end=11.25)]
exit_points = [(150.8, -53.0), (136.4, -45.0), (125.0, -68.5), (146.2, -73.8)]
print("Generated Location 2 Topology...")
return Topology(roundabout_center, circular_lanes, exit_points)
# exit_points must be given in trigonometric order starting from the smallest angle.
......@@ -352,7 +368,7 @@ class Topology:
(xe, ye) = draw_circle(point, self.exits_radius)
plt.plot(xe, ye, color="red")
plt.text(point[0], point[1], "{}".format(ix), color='white', ha='center', va='center')
plt.text(point[0], point[1], "{}".format(ix), color='grey', ha='center', va='center')
# Utils
......
......@@ -234,6 +234,9 @@ class TTCData:
@staticmethod
def compute_quartile_coefficient_of_dispersion(series):
if len(series) == 0:
return -1.0
Q3 = np.quantile(series, .75)
Q1 = np.quantile(series, .25)
return (Q3-Q1)/(Q3+Q1)
......
......@@ -23,9 +23,9 @@ from ttc_correlation import TTCTimeline, TTCData, VariationDataset
# TODO: 1. DEFINE THE PATH TO A TRACKS-FILE REPRESENTING THE WANTED LOCATION HERE
# Note: E.G for a tracks of location ID=0 in my case.
input_file = "09_tracks.csv"
input_meta_file = "09_tracksMeta.csv"
input_recordingMeta_file = "09_recordingMeta.csv"
input_file = "00_tracks.csv"
input_meta_file = "00_tracksMeta.csv"
input_recordingMeta_file = "00_recordingMeta.csv"
# .......................... #
# A. Loading the tracks file #
......@@ -82,12 +82,13 @@ def plot_tracks(tracks_range, draw_lanes, topology):
# .................................................................. #
# TODO: 2. DEFINE THE TOPOLOGY OF THE LANES FOR THAT SPECIFIC ROUNDABOUT
topology = Topology.roundDLocation0Topology() # see topology.py
topology = Topology.roundDLocation1Topology() # see topology.py
# Plot all the tracks trajectories from the given tracks file
#plot_tracks(range(0, last_track+1), True, topology)
#plt.show()
plot_tracks(range(0, last_track+1), True, topology)
plt.show()
'''
# ................................................................ #
# C. This is a step by step visualzation tool to replay the tracks #
......@@ -118,7 +119,7 @@ def read_frames(last_frame=-1, last_track=-1):
print ("Loading frames...")
# 2000 frames only loaded, so that the script launches faster.
# last_frame=last_frame for full visualization
frames = read_frames(last_frame=200, last_track=last_track)
frames = read_frames(last_frame=10000, last_track=last_track)
def draw_object(ax, x, y, width, height, heading):
rect = patches.Rectangle((x-width/2, y-height/2), width, height, linewidth=1, edgecolor='r', facecolor='r')
......@@ -126,22 +127,23 @@ def draw_object(ax, x, y, width, height, heading):
rect.set_transform(transform)
return rect
fig, ax = plt.subplots()
# NOTE: background image of the roundabout.
# Needs extra work as it must be manually positioned to match the tracks
# of each vehicles
# The provided example shows the background of location ID=0
im = Image.open('loc0_background.png')
#im = Image.open('loc1_background.png')
for frameId in range(0, len(frames), 1):
for frameId in range(50, len(frames), 10):
plt.title("Frame {}/{}".format(frameId, len(frames)))
ax.set_xlim(35,125)
ax.set_ylim(-94,-1)
ax.set_xlim(60,160)
ax.set_ylim(0,-140)
# NOTE: manual positioning of the background image
ax.imshow(im, extent=[1,169,-94,-1])
#ax.imshow(im, extent=[51.7,171.7,-164,26])
topology.draw()
......@@ -158,4 +160,4 @@ for frameId in range(0, len(frames), 1):
plt.waitforbuttonpress()
plt.cla()
'''
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment