INF 554 Project

PROJECT SUMMARY

PROJECT INFORMATION

• Project title: Traffic Collision Visualization
• Group name: Team Owl
• Team members:
  • Bufan Zeng ([email protected])
  • Wing Wa Leung ([email protected])
  • Yunwei Zhang ([email protected])

PROJECT ARTIFACTS

• Demonstration URL
• Presentation PDF and transcript
• Article and Overleaf URL
• YouTube video

PROJECT DEVELOPMENT

1. Install Angular CLI, Bootstrap, jQuery, Popper.js and D3 with npm

• Node.js and Angular CLI has to be installed globally beforehand
• Open the terminal, move to the directory one level above the GitHub local repository
• To create a new project, type in the command ng new project-owl
• Move to the GitHub local repository
• To install Bootstrap, jQuery, Popper.js, D3 and ts types declaration for D3, type in the command npm install bootstrap jquery popper.js d3 @types/d3 --save
• Edit angular.json to link Bootstrap core CSS, D3, jQuery, Popper.js and Bootstrap

"styles": [
    "src/styles.css",
    "node_modules/bootstrap/dist/css/bootstrap.min.css"
],
"scripts": [
    "node_modules/d3/dist/d3.min.js",
    "node_modules/jquery/dist/jquery.slim.min.js",
    "node_modules/popper.js/dist/umd/popper.min.js",
    "node_modules/bootstrap/dist/js/bootstrap.min.js"
]

2. Data acquirement

2.1 Traffic Collision Data

• Search the dataset "Traffic Collision Data from 2010 to Present" in Los Angeles Open Data
• Open the dataset, click "Export" then choose "CSV"
• Download the dataset as a CSV (comma-separated values) file

2.2 Los Angeles County map

• Find the map "L.A. County Neighborhoods (V6)" in Mapping L.A. Boundaries API provided by Los Angeles Times
• Right click "GeoJSON" button, and save the link as GeoJSON file
• Move the file to /src/assets folder
• Edit angular.json to link the GeoJSON

3. Data cleaning and feature engineering

3.1 Data cleaning

• The project aims to visualize the data using different perspectives and the data itself is rather big for D3 to load as whole. Individual data cleaning for different charts are required.
• The most frequently used technique is to check the distribution of the required attributes and remove the N/A or "unknown" data points.
• For specific charts, pivot table is used and noise are further removed (noisy levels in attributes.)

3.3 Feature engineering

3.3.1 Line chart

• Extract the month and year of the accidents and then calculate the accumulated number of accidents for each month of 9 years.

3.3.2 Choropleth maps

• To have the same features that can enable us to map the data using GeoJson file, the Python library "uszipcode" is used.
• Query using the library API one record at a time using the latitude and longitude attributes from data set. The whole process took more than 2 hours to complete.
• Extract the rich information from the API call results, specifically, the "major_city" attribute and write the dataframe back with this additional feature.
• Add a time attribute to the dataframe by recoding the "time occurred" attribute into 4 categories - daytime, morning peak, nighttime, evening peak.
• Pivot the data using the two new attributes with the accumulated count.
• For small multiples, calculate the average hourly percentage for each city by dividing deach time period by its number of hours. Then calculate the sum of each city and then convert the data for each time period into percentage.

3.3.3 Slopegraphs

• The slopegraph uses the hourly cumulative count of each city's accidents with respect to the time period mentioned above.

3.3.4 Bar chart

• The data for this chart requires gender and age groups, which the gender is provided in the raw data. Only the "M" and "F" records are used.
• The age groups are generated by grouping the age attribute into 10 years each. For example, the 10-20 years old victims will be grouped into one.

3.3.5 Bubble cloud

• For each descent, count the number of victims, save the results as CSV file
• For each row in the CSV file, add a new attribute "parent" with same value "Root"
  • For the sake of circle packing layout, each row has to have a parent
• Add a new row for "Root"

3.3.6 Alluvial diagram

• The data for this diagram consists of four attributes - the age group and gender from bar chart, the time period from the choropleth map and the accident location from the cleaned data set.

3.4 Configure angular.json

• Edit angular.json to link the datasets

4. Build page layout

• In /src/index.html, change page title
• In /src/app/app.component.html, use Bootstrap to:
  • Create a navigation bar for displaying group name
  • Create a <div> element of class "container-fluid" for full width container
  • Build page layout with grid system
  • Add texts to introduce the data and the charts
• In /src/app/app.component.css, include customized style attributes
• To preview the website, type in the command ng serve --open in the terminal

5. Plot interactive and responsive line chart

• To create an Angular component, type the command ng generate component line-chart in the terminal
• In /src/app/app.component.html, add <app-line-chart></app-line-chart> to link the component to the main page
• In /src/app/line-chart/line-chart.component.html:
  • Use <h3> element to write chart name and use <p> element to write user instructions to interact with the chart
  • Use <div> element to make an id to call in line-chart.component.ts
• In /src/app/line-chart/line-chart.component.css:
  • Define styles for axis, lines, tooltips and legends
• In /src/app/line-chart/line-chart.component.ts, import D3
• In function ngOnInit():
  • Browser window will be selected for later responsive effect (if smaller screen, yearly header legend not show)
  • Set width and height and margin with width responsive effect; draw axis and lines
  • There are ids set for linking header legends with lines so that click header legends would hide/show lines.
  • Also on(click) for data points interaction

6. Plot interactive choropleth maps with small multiples

• To create an Angular component, type the command ng generate component choropleth-map in the terminal
• In /src/app/app.component.html, add <app-choropleth-map></app-choropleth-map> to link the component to the main page
• In /src/app/choropleth-map/choropleth-map.component.html:
  • Create a <div> element of class "container" and id "chart2"
  • Create 5 <svg> elements of ids "map", "map1", "map2", "map3", "map4" inside "#chart2"
  • Build chart layout with grid system
• In /src/app/choropleth-map/choropleth-map.component.css,define style attributes for "#chart2"
• In /src/app/choropleth-map/choropleth-map.component.ts, import D3
• In function ngOnInit(), load dataset for choropleth maps using d3.csv(), and after data is loaded:
  • Define width and height of the major choropleth map, as well as margins for displaying the title and legend
    • The total available width (including margins) for the map is the half of d3.select("#chart2").style("width"); height equals to d3.select("#chart2").style("height")
  • Select "#map" and set the svg element up using margin convention
  • Define width and height of each of the small multiples, as well as margins for displaying the title
    • The total available width (including margins) for the map is the one-fourth of d3.select("#chart2").style("width"); height is half of d3.select("#chart2").style("height")
  • Select "#map1", "#map2", "#map3" and "#map4" separately and set each svg element up using margin convention
  • For each chart, add title
  • Divide the data into 5 hashmaps, each uses city name as key and the cumulative accident frequency during specific time period as value
  • Define a color scheme for map using d3.scaleQuantile()
    • Since the variation of total number of accidents is large, "quantile scale" is preferred to encode color scheme over "quantize scale"
    • Rather than using sequential colors, it is easier to distinguish the severity level with the use of diverging color scheme
  • Define a color scehme for the small multiples using d3.scaleOrdinal()
    • It is similar to the one for map; however, the data presented are not absolute counts, instead we used the relative percentage and then ranked them with respect of each city so that the change (trend) can be more obvious than using the absolute count
  • For the major choropleth map, add legend
  • Create a projection for map using d3.geoMercator()
  • Create a projection for the small multiples using d3.geoMercator()
  • For each projection, create a geographic path generator for path using d3.geoPath()
  • Using D3 data join with data json["features"]:
    • Append <path>s to the svg element, each is a city's boundary
    • Specify the fill style attribute according to the cumulative accident frequency
    • For each <path> element appended to map only, attach mouse-over and mouse-out event listeners and append <title> as tooltip to show the cumulative accident frequency of the corresponding city

7. Plot interactive slopegraphs

• To create an Angular component, type the command ng generate component slopegraph in the terminal
• In /src/app/app.component.html, add <app-slopegraph></app-slopegraph> to link the component to the main page
• In /src/app/slopegraph/slopegraph.component.html, use Bootstrap to:
  • Create a <div> element of class "container" and id "chart3"
  • Create 2 <svg> elements of ids "slopegraph1" and "slopegraph2" inside "#chart3"
  • Build chart layout with grid system to display the charts side by side
• In /src/app/slopegraph/slopegraph.component.css, define style attributes for "#chart3"
• In /src/app/slopegraph/slopegraph.component.ts, import D3
• In function ngOnInit(), load dataset for slopegraph using d3.csv(), and after data is loaded:
  • Define width and height of the each slopegraph, as well as margins for displaying the title and legend
    • The total available width (including margins) for each chart is the half of d3.select("#chart3").style("width"); similarly height is the half of d3.select("#chart3").style("height")
  • Select "#slopegraph1" and set the svg element up using margin convention
  • Select "#slopegraph2" and set the svg element up using margin convention
  • For each chart, add title and legend
  • Divide the data into 2 parts, one is related to Los Angeles only, another is related to the rest of the cities
    • As the discrepancy of data between Los Angeles and the rest is very large, separate scales would be used
  • Define variables scale_la and scale_others for the scales of data related to Los Angeles and the rest of the cities respectively using d3.scaleLinear()
  • Using D3 data join:
    • Append labels of class "label_city_name" with same x-coordinate (e.g. -40) to slopegraph1; the text of each label is a city name, and its y-coordinate is the scaled cumulative accident frequency per hour during morning peak
    • Append labels of class "text1_city_name" with same x-coordinate (e.g. 0) to slopegraph1; the text of each label is cumulative accident frequency per hour during morning peak, and its y-coordinate is the scaled frequency
    • Append labels of class "text2_city_name" with same x-coordinate (e.g. width_sg) to slopegraph1; the text of each label is cumulative accident frequency per hour during evening peak, and its y-coordinate is the scaled frequency
    • Append lines of class "line_city_name" to slopegraph1; and for each city:
      • x1: padding_sg
      • y1: scaled cumulative accident frequency per hour during morning peak
      • x2: width_sg - padding_sg
      • y2: scaled cumulative accident frequency per hour during evening peak
    • Append labels of class "label_city_name" with same x-coordinate (e.g. -40) to slopegraph2; the text of each label is a city name, and its y-coordinate is the scaled cumulative accident frequency per hour during daytime
    • Append labels of class "text1_city_name" with same x-coordinate (e.g. 0) to slopegraph2; the text of each label is cumulative accident frequency per hour during daytime, and its y-coordinate is the scaled frequency
    • Append labels of class "text2_city_name" with same x-coordinate (e.g. width_sg) to slopegraph2; the text of each label is cumulative accident frequency per hour during nighttime, and its y-coordinate is the scaled frequency
    • Append lines of class "line_city_name" to slopegraph2; and for each city:
      • x1: padding_sg
      • y1: scaled cumulative accident frequency per hour during daytime
      • x2: width_sg - padding_sg
      • y2: scaled cumulative accident frequency per hour during nighttime
  • Attach mouse-over and mouse-out event listenters to all elements

8. Plot bar chart with animated transitions

• To create an Angular component, type the command ng generate component age-chart in the terminal
• In /src/app/app.component.html, add <app-age-chart></app-age-chart> to link the component to the main page
• In /src/app/age-chart/age-chart.component.html:
  • Create a list of button group
  • Create a <div> element of id "chart4"
  • Inside "#chart4":
    • Create a <svg> element of id "age-gender-chart" for the bar chart
    • Create a <div> element of class "hidden" and id "tooltip" for the tooltip
    • Inside "#tooltip", create a <p> element of id "tooltip_text" for the tooltip's text contents
• In /src/app/age-chart/age-chart.component.css, define style attributes for "#tooltip" and "#tooltip.hidden"
• In /src/app/age-chart/age-chart.component.ts, import D3
• In function ngOnInit():
  • Read in the data as csv.
  • Create initial state, which is 10-20 years old. Do the data join by reading in the specific data column, which has 2 genders * 9 years = 18 rows.
  • Create axises, the yAxis (scaleLinear) is set to the maximum number of age group 21-30 because this age group has most accidents. Create xAxis (scaleBand) set to be fixed with 9 years.
  • Draw the bars by checking if the specific data entry is male or not, if it's male, use the scaled data directly for "y" in rect element. If it's female, the "y" should be the scaled data plus the one of male so they can stack.
  • Define the transition function by reading the corresponding age groups that are linked to the buttons at top. When on click, the parameter will be passed into variable and the function will check if the current state is the same as the one clicked. If not, do the transition animation by repeating the previous bar drawing process with the new age group.
  • If the age groups that beyond 70 are choosen, redraw the yAxis by using the maximum count of "71-80" age group. The reason is that the number of accidents beyond age 70 is very small comparing to the younger groups and the data would be hard to be observed using the previous scale.
  • Judge if the yAxis needs to change by defining an axis transition function.
  • Add tooltips for each rectangle and add legends.

9. Plot interactive bubble cloud

• To create an Angular component, type the command ng generate component bubble-cloud in the terminal
• In /src/app/bubble-cloud/bubble-cloud.component.html:
  • Create a <div> element of id "chart5"
  • Inside "#chart5":
    • Create a <svg> element of id "bubble_cloud" for the bubble cloud
    • Create a <div> element of class "hidden" and id "tooltip1" for the tooltip
    • Inside "#tooltip1", create a <p> element of id "tooltip_text1" for the tooltip's text contents
• In /src/app/bubble-cloud/bubble-cloud.component.css, define style attributes for "#tooltip1" and "#tooltip1.hidden"
• In /src/app/app.component.html, add <app-bubble-cloud></app-bubble-cloud> to link the component to the main page
• In /src/app/bubble-cloud/bubble-cloud.component.ts, import D3
• Define an interface for each row's data structure in the CSV file
• In function ngOnInit(), load the dataset for bubble cloud using d3.csv(), and after data is loaded:
  • Define data's hierarchy using d3.stratify()
  • Define width and height of the chart, as well as margins for displaying the chart title
  • Select "#bubble_cloud" and set the svg element up using margin convention
  • Add chart title
  • Convert data to the hierarchy
  • Create a circle packing layout with d3.pack()
  • Define a color scheme using d3.scaleOrdinal()
  • Define a scale for font size using d3.scaleLinear()
  • Using D3 data join:
    • Filter out the root element with d3.filter()
    • Append <circle>s to the svg element, each represents a descent
    • Append <text>s to the svg element, each labels the bubble's corresponding descent
    • Attach 2 event listeners to each element:
      • When the user hovers the mouse on the bubble:
        • Select "#tooltip1", and change its top and left positions
        • Select "#tooltip_text1", and change its text to the corresponding descent's information
        • Remove class "hidden" from "#tooltip1"
      • When the user moves the mouse out of the bubble:
        • Add class "hidden" to "#tooltip1"

10. Plot interactive and responsive alluvial diagram

• To create an Angular component, type the command ng generate component alluvial-diagram in the terminal
• In /src/app/app.component.html, add <app-alluvial-diagram></app-alluvial-diagram> to link the component to the main page
• This diagram is generated by a third party application named RAWGraphs. Simply filter out the desired attributes and records from original data, upload it to their server and then follow the prompts, the alluvial diagram will be generated in svg form.
• Insert the generated svg to the component HTML and assign an "id".
• Used D3 in typescript to add the mouseover functions for each path.

11. Publish the website to USC SCF

• To deploy the website, type in the command ng build --prod --output-path deploy --base-href http://www-scf.usc.edu/~wingwale/project-owl/ in the terminal
• Connect to one of USC SCF's servers (aludra.usc.edu or nunki.usc.edu) with SFTP (secure FTP) using Filezilla
• Create a new directory "project-owl" under "public_html"
• Copy all local files in "/deploy" to "public_html/project-owl"