Config Settings

Params of Env Simulator

This will introduce the details about environmental simulators’ Parameters

CityFlow Config File

Referred from: https://cityflow.readthedocs.io/en/latest/start.html#data-access-api

{
  "interval": 1.0,
  "seed": 0,
  "dir": "data/",
  "roadnetFile": "cityflow_1x1/roadnet.json",
  "flowFile": "cityflow_1x1/flow.json",
  "rlTrafficLight": true,
  "saveReplay": false,
  "roadnetLogFile": "cityflow_1x1/replay/2022_07_19-09_28_40.json",
  "replayLogFile": "cityflow_1x1/replay/2022_07_19-09_28_40.txt"
}

• interval: time of each simulation step (in seconds).

• seed: random seed.

• dir: root directory, all file path will be relative to this directory.

• roadnetFile: path for roadnet file.

• flowFile: path for flow file.

• rlTrafficLight: whether to enable traffic light control through python API. If set to false, default traffic light plan defined in roadnetFile will be used.

• saveReplay: whether to save simulation for replay. If set to true, roadnetLogFile and replayLogFile are required.

• roadnetLogFile: path for roadnet replay file. This is a special roadnet file for replay, not the same as roadnetFile.

• replayLogFile: path for replay. This file contains vehicle positions and traffic light situation of each simulation step.

SUMO Config File

{
  "interval": 1.0,
  "seed": 0,
  "dir": "data/",
  "combined_file": "cityflow_1x1/cityflow_1x1.sumocfg",
  "roadnetFile": "cityflow_1x1/cityflow_1x1.net.xml",
  "flowFile": "cityflow_1x1/cityflow_1x1.rou.xml",
  "convertroadnetFile": "cityflow_1x1/cityflow_1x1_roadnet.json",
  "convertflowFile": "cityflow_1x1/cityflow_1x1_flow.json",
  "no_warning": true,
  "name": "debug",
  "yellow_length": 5,
  "gui": false
}

• interval: time of each simulation step (in seconds).

• seed: random seed.

• dir: root directory, all file path will be relative to this directory.

• combined_file: path for SUMO configuration file.

• roadnetFile: path for SUMO roadnet file.

• flowFile: path for SUMO flow file.

• convertroadnetFile: path for SUMO roadnet file.

• convertflowFile: path for SUMO flow file.

• no_warning:

• name:

• yellow_length: default yellow time. The default is set to 5 s.

• gui: whether to use SUMO-GUI tools.

Params of TSC Process and Agents

Parameters about training proccess and agents

base.yml

common parameters of the framework

task:
  description: Use RL methods to optimize traffic signal control problem
  task_name: tsc
world:
# necessary and distinctive parameters for simulators
  interval: 1.0
  seed: 0
  dir: data/

  saveReplay: False
  report_log_mode: normal # cbengine
  report_log_rate: 10

  no_warning: True
  gui: False

  rlTrafficLight: True

trainer:
  thread: 4
  ngpu: -1
  learning_start: 5000
  buffer_size: 5000
  steps: 3600
  test_steps: 3600
  yellow_length: 5
  action_interval: 10
  episodes: 200
  update_model_rate: 1
  update_target_rate: 10
  test_when_train: True

model:
  name: "non-rl"
  train_model: False
  test_model: True
  load_model: False
  graphic: False
  vehicle_max: 1
  learning_rate: 0.001
  batch_size: 64
  gamma: 0.95
  epsilon: 0.5
  epsilon_decay: 0.99
  epsilon_min: 0.05
  grad_clip: 5.0

  one_hot: False
  phase: False

logger:
  root_dir: "data/output_data/"
  log_dir: "logger/"
  replay_dir: "replay/"
  model_dir: "model/"
  data_dir: "dataset/"
  save_model: True
  save_rate: 200
  attention: False

• Task

  • description: description of task

  • task_name: task name

• World

  • interval: time of each simulation step (in seconds).

  • seed: random seed.

  • dir: root directory, all file path will be relative to this directory.

  • saveReplay: whether to save simulation for replay. Used for CityFlow.

  • report_log_mode:

  • report_log_rate: rate of report log.

  • no_warning:

  • gui: whether use SUMO-GUI tools

  • rlTrafficLight: whether to enable traffic light control through python API. If set to false, default traffic light plan defined in roadnetFile will be used.

• Trainer

  • thread: number of threads.

  • ngpu: number of gpu to be used.

  • learning_start: how many steps start to update model.

  • buffer_size: size of memory buffer.

  • steps: number of training steps in one episode.

  • test_steps: number of test steps in one episode.

  • yellow_length: yellow time.

  • action_interval: time interval to perform actions.

  • episodes: number of one complete process.

  • update_model_rate: rate of updating model.

  • update_target_rate: rate of updating target model.

  • test_when_train: whether to test model after each episode.

• Model

  • name: type of model, support non-rl and rl models.

  • train_model: whether to train model, False for non-rl models, True for rl models.

  • test_model: whether to test model.

  • load_model: whether to load pretrained model.

  • graphic: whether to use graph to train model. Now it support DGL.

  • vehicle_max:

  • learning_rate: the step size at each iteration while moving toward a minimum of a loss function.

  • batch_size: the number of samples utilized in one iteration.

  • gamma: discount factor.

  • epsilon: probability of random actions.

  • epsilon_decay: decay of epsilon on the next training episode.

  • epsilon_min: minimum value of epsilon.

  • grad_clip: clip gradients according to this parameter.

  • one_hot: whether to encode phase.

  • phase: whether to add phase into model’s feature.

• Logger

  • root_dir: root directory, all file path of output data will be relative to this directory.

  • log_dir: root directory of logger.

  • replay_dir: root directory of replay.

  • model_dir: root directory of model.

  • data_dir: root directory of dataset.

  • save_model: whether to save model’s parameters.

  • save_rate: rate of saving related parameters.

  • attention:

Note: The value of common parameters will be overwritten if it appears in a model file when loading parameters in training process.

FixedTime.yml

includes:
  - configs/tsc/base.yml

model:
  name: fixedtime
  t_fixed: 10

trainer:
  episodes: 1
  buffer_size: 0
  action_interval: 10
  learning_rate: 0
  learning_start: 0
  update_model_rate: 0
  update_target_rate: 0

logger:
  save_rate: 1
  train_model: False
  test_model: True
  load_model: False

MaxPressure.yml

includes:
  - configs/tsc/base.yml

model:
  name: maxpressure
  t_min: 10

trainer:
  buffer_size: 0
  action_interval: 10
  learning_rate: 0
  learning_start: 0
  update_model_rate: 0
  update_target_rate: 0

logger:
  save_rate: 0
  train_model: False
  test_model: True
  load_model: False

• t_min:

SOTL.yml

includes:
  - configs/tsc/base.yml

model:
  name: sotl
  min_green_vehicle: 3
  max_red_vehicle: 6
  t_min: 5

trainer:
  episodes: 1
  buffer_size: 0
  learning_rate: 0
  learning_start: 0
  update_model_rate: 0
  update_target_rate: 0

logger:
  save_rate: 0
  train_model: False
  test_model: True
  load_model: False

IDQN.yml

includes:
  - configs/tsc/base.yml

model:
  name: dqn
  train_model: True

MAPG.yml

includes:
  - configs/tsc/base.yml

model:
  name: magd
  train_model: True
  local_q_learn: False
  tau: 0.01
  learning_rate: 0.01
  batch_size: 256
  grad_clip: 0.5
  epsilon: 0.5
  epsilon_decay: 0.998
  epsilon_min: 0.05

trainer:
  episodes: 2000
  update_model_rate: 30
  update_target_rate: 30
  action_interval: 20

IPPO.yml

includes:
  - configs/tsc/base.yml

model:
  name: ppo
  graphic: False
  vehicle_max: 1
  learning_rate: 0.001
  update_interval: 1024
  batch_size: 64
  gamma: 0.95
  epsilon: 0.1
  epsilon_decay: 0.995
  epsilon_min: 0.01
  grad_clip: 5.0
  NEIGHBOR_NUM: 4
  NEIGHBOR_EDGE_NUM: 4

trainer:
  learning_start: 1000
  buffer_size: 1024

PressLight.yml

includes:
  - configs/tsc/base.yml

model:
  name: presslight
  train_model: True
  d_dense: 20
  epsilon: 0.1
  epsilon_decay: 0.995
  epsilon_min: 0.01
  one_hot: True
  phase: True

trainer:
  learning_start: 1000

CoLight.yml

includes:
  - configs/tsc/base.yml

model:
  name: colight
  graphic: True
  vehicle_max: 1
  learning_rate: 0.001
  batch_size: 64
  gamma: 0.95
  epsilon: 0.8
  epsilon_decay: 0.9995
  epsilon_min: 0.01
  grad_clip: 5.0
  NEIGHBOR_NUM: 4
  NEIGHBOR_EDGE_NUM: 4
  N_LAYERS: 1
  INPUT_DIM: [128, 128]
  OUTPUT_DIM: [128, 128]
  NODE_EMB_DIM: [128, 128]
  NUM_HEADS: [5, 5]
  NODE_LAYER_DIMS_EACH_HEAD: [16, 16]
  OUTPUT_LAYERS: []
  one_hot: True
  phase: False

trainer:
  learning_start: 1000
  train_model: True

FRAP.yml

includes:
  - configs/agents/tsc/base.yml

model:
  name: frap
  n_layers: 2
  rotation: true
  conflict_matrix: true
  merge: multiply
  d_dense: 20
  learning_rate: 0.001
  batch_size: 64
  gamma: 0.95
  epsilon: 0.1
  epsilon_decay: 0.995
  epsilon_min: 0.01
  grad_clip: 5.0
  demand_shape: 1
  one_hot: False
  phase: True

trainer: 
  learning_start: 1000
  train_model: True

world:
  phases: ['NT_ST','WT_ET','NL_SL','WL_EL','NL_NT','SL_ST','WL_WT','EL_ET']
  
  list_lane_order: ['ET','EL','ST','SL','WT','WL','NT','NL']

  phase_expansion: {
    1: [0, 0, 1, 0, 0, 0, 1, 0],
    2: [1, 0, 0, 0, 1, 0, 0, 0],
    3: [0, 0, 0, 1, 0, 0, 0, 1],
    4: [0, 1, 0, 0, 0, 1, 0, 0],
    5: [0, 0, 0, 0, 0, 0, 1, 1],
    6: [0, 0, 1, 1, 0, 0, 0, 0],
    7: [0, 0, 0, 0, 1, 1, 0, 0],
    8: [1, 1, 0, 0, 0, 0, 0, 0]
  }

  phase_expansion_4_lane: {1: [0,0,1,1],2: [1,1,0,0]}

  signal_config: {
    grid4x4: {
      phase_pairs: [[1, 7], [2, 8], [1, 2], [7, 8], [4, 10], [5, 11], [10, 11], [4, 5]],
      valid_acts: null,
      cf_order: {'N': 0,'E': 1,'S': 2, 'W': 3},
      sumo_order: {'N': 2,'E': 3,'S': 0, 'W': 1}
    },
    hz1x1: {
      # phases: ['ET_WT','NT_ST','EL_WL','NL_SL','WL_WT','EL_ET','SL_ST','NL_NT']
      phase_pairs: [[2, 6], [0, 4], [3, 7], [1, 5], [6, 7], [2, 3], [4, 5], [0, 1]],
      # phases: ['NT_ST','WT_ET','NL_SL','WL_EL','NL_NT','SL_ST','WL_WT','EL_ET']
      # phase_pairs: [[1, 5], [3, 7], [0, 4], [2, 6], [0, 1], [4, 5], [6, 7], [2, 3]],
      valid_acts: null,
      cf_order: {'N': 0,'E': 1,'S': 2, 'W': 3},
      sumo_order: {'N': 2,'E': 3,'S': 0, 'W': 1}
    },
    hz4x4: {
      # phases: ['NT_ST','WT_ET','NL_SL','WL_EL','NL_NT','SL_ST','WL_WT','EL_ET']
      phase_pairs: [[4, 10], [1, 7], [3, 9], [0, 6], [9, 10], [3, 4], [6, 7], [0, 1]],
      valid_acts: null,
      cf_order: {'N': ,'E': ,'S': , 'W': },
      sumo_order: {'N': ,'E': ,'S': , 'W': }
    },
    cologne1: {
      phase_pairs: [[1, 5], [0, 4], [3, 7], [2, 6]],
      valid_acts: null,
      cf_order: {'N': 3,'E': 0,'S': 1, 'W': 2},
      sumo_order: {'N': 1,'E': 2,'S': 3, 'W': 0}
    },
    cologne3: {
      phase_pairs: [[1, 7], [2, 8], [1, 2], [7, 8], [4, 10], [5, 11], [10, 11], [4, 5], [9, 11]],
      valid_acts: {
        GS_cluster_2415878664_254486231_359566_359576: {4: 0, 5: 1, 0: 2, 1: 3},
        360086: {4: 0, 5: 1, 0: 2, 1: 3},
        360082: {4: 0, 5: 1, 1: 2}
      },
      cf_order: {
        '360082': {'N': -1,'E': 0,'S': 1, 'W': 2},
        '360086': {'N': 3,'E': 0,'S': 1, 'W': 2},
        'cluster_2415878664_254486231_359566_359576': {'N': 3,'E': 0,'S': 1, 'W': 2}
        },
      sumo_order: {
        '360082': {'N': -1,'E': 1,'S': 2, 'W': 0},
        '360086': {'N': 1,'E': 2,'S': 3, 'W': 0},
        'cluster_2415878664_254486231_359566_359576': {'N': 1,'E': 2,'S': 3, 'W': 0}
      }
    }
  }

MPLight.yml

includes:
  - configs/tsc/base.yml

model:
  name: mplight
  train_model: True
  n_layers: 2
  rotation: true
  conflict_matrix: true
  merge: multiply
  d_dense: 20
  learning_rate: 0.001
  batch_size: 32
  gamma: 0.99
  eps_start: 1.0
  eps_end: 0.0
  eps_decay: 220
  target_update: 500
  demand_shape: 1
  phase: True

trainer: 
  learning_start: -1
  buffer_size: 10000

world:
  phases: ['NT_ST','WT_ET','NL_SL','WL_EL','NL_NT','SL_ST','WL_WT','EL_ET']
  
  list_lane_order: ['ET','EL','ST','SL','WT','WL','NT','NL']

  phase_expansion: {
    1: [0, 0, 1, 0, 0, 0, 1, 0],
    2: [1, 0, 0, 0, 1, 0, 0, 0],
    3: [0, 0, 0, 1, 0, 0, 0, 1],
    4: [0, 1, 0, 0, 0, 1, 0, 0],
    5: [0, 0, 0, 0, 0, 0, 1, 1],
    6: [0, 0, 1, 1, 0, 0, 0, 0],
    7: [0, 0, 0, 0, 1, 1, 0, 0],
    8: [1, 1, 0, 0, 0, 0, 0, 0]
  }

  phase_expansion_4_lane: {1: [0,0,1,1],2: [1,1,0,0]}

  signal_config: {
    grid4x4: {
      phase_pairs: [[1, 7], [2, 8], [1, 2], [7, 8], [4, 10], [5, 11], [10, 11], [4, 5]],
      valid_acts: null
    },
    cityflow1x1: {
      # phases: ['ET_WT','NT_ST','EL_WL','NL_SL','WL_WT','EL_ET','SL_ST','NL_NT']
      # phase_pairs: [[3, 7], [1, 5], [2, 6], [0, 4], [6, 7], [2, 3], [4, 5], [0, 1]],
      # phases: ['NT_ST','WT_ET','NL_SL','WL_EL','NL_NT','SL_ST','WL_WT','EL_ET']
      phase_pairs: [[1, 5], [3, 7], [0, 4], [2, 6], [0, 1], [4, 5], [6, 7], [2, 3]],
      valid_acts: null
    },
    cityflow4x4: {
      # phases: ['NT_ST','WT_ET','NL_SL','WL_EL','NL_NT','SL_ST','WL_WT','EL_ET']
      phase_pairs: [[4, 10], [1, 7], [3, 9], [0, 6], [9, 10], [3, 4], [6, 7], [0, 1]],
      valid_acts: null
    },
    cologne1: {
      phase_pairs: [[1, 7], [2, 8], [4, 10], [5, 11]],
      valid_acts: null
    },
    cologne3: {
      phase_pairs: [[1, 7], [2, 8], [1, 2], [7, 8], [4, 10], [5, 11], [10, 11], [4, 5], [9, 11]],
      valid_acts: {
        GS_cluster_2415878664_254486231_359566_359576: {4: 0, 5: 1, 0: 2, 1: 3},
        360086: {4: 0, 5: 1, 0: 2, 1: 3},
        360082: {4: 0, 5: 1, 1: 2}
      }
    }
  }