A microscopic simulation model is run multiple times for the roadway segment and traffic period being analysed in order to determine the mean values that are needed to analyse transportation design alternatives. To save money and time, engineers often elect to reduce the number of runs in microsimulations. However, this can lead to unreliable results. This article describes a Federal Highway Administration study that investigated the relationship between the number of traffic simulation runs and the aggregate results under various levels of error. The goal was to determine the level of simulation required for a calibrated network -- a model that reproduces field-measured traffic conditions -- to be considered statistically significant at a predefined confidence level for various measures of effectiveness (MOEs). FHWA's Corridor Simulation (CORSIM) software was used in a case study of computer model runs. The study used a calibrated model of six freeway links in the Minneapolis/St. Paul metropolitan area. Researchers analysed the six links for the afternoon peak period. They calculated three link-level MOEs -- lane density, link discharge, and link speed -- for each link by analysing the measured data collected during the CORSIM simulation runs. They also calculated the sampling error for the mean value at 95% confidence for each MOE. A sampling error under 10% represented stability in the mean value. For each of the six freeway links, the researchers carried out six independent run sets (5, 10, 15, 20, 25, and 30). Many of the numerical and graphical representations of the run sets revealed recognizable improvements to the results between 5 and 10 runs. Overall evidence indicated that at least 10 to 15 simulation runs were needed to stabilize the results. After the researchers achieved a stabilized run set, minimal benefit was obtained from additional runs. The sampling error peaked at over 70% for the 5-run set during the queue formation but was reduced to 20% for the 30-run set. The improvements to the mean values did vary between run sets. The researchers also documented the level of effort required to run the sets for the six scenarios, noting the time required for additional runs, the total time to complete all runs and analysis, the average time per run, and the range of times per run for each set. Results showed that the level of effort for additional runs was not significant. (Author/publisher)
Abstract