Week 47 Β· October 20, 2026

Behavior Metrics: Right-Lane Percentage and Steering Zero-Crossings per Second β€” First PilotNet Run in Town02

October 20, 2026

πŸ“Š Part 1 β€” Overview: Two New Metrics Added to Behavior Metrics

πŸ“Œ What Was Added

Two per-experiment metrics were added to Behavior Metrics: the percentage of frames in the right lane (right_lane_percentage) and the steering zero-crossings per second (steering_zero_crossings_per_second). Both are computed in the Python API backend from the CSV that the controller writes at every CARLA tick, and are stored in the experiment's summary JSON.

The process has two halves:

1. During recording (utils/controller_carla.py)

At each simulator tick, one CSV row is written. The patch adds five columns describing the lane the ego vehicle is in at that instant.

2. At recording stop (utils/metrics_carla.py)

get_metrics_python_api() reads the CSV and now calls _bm_extra_metrics(), which aggregates those columns together with the already existing steer column into the two metrics.

As extras, the summary JSON also keeps off_driving_lane_percentage, steering_zero_crossings (the total), and steering_zero_crossings_deadband (the deadband used).

Flow diagram: from CARLA tick to the JSON summary
New metrics travel from the CARLA tick to the JSON. Top: at each tick. Bottom: when recording stops. Blue: patch code.

πŸ“Œ Why Python API and Not ROS

  • No ROS on this machine. The team uses Ubuntu 26.04.1 and ROS is not installed. The poetic-devel branch of Behavior Metrics and its carla-ros-bridge target ROS 1 Noetic, which corresponds to Ubuntu 20.04. Without ROS there is no bridge, no /carla/ego_vehicle/* topics, and no rosbags.
  • The ROS backend depends on rosbags. With ROS_VERSION set to 1 or 2, ControllerCarla.record_metrics() launches rosbag record / ros2 bag record on the bridge topics, and metrics_carla.get_metrics() processes the bag. None of that can run here.
  • The Python API backend already existed and is the one that runs. With ROS_VERSION=None, the controller subscribes to world.on_tick, writes the CSV with _sample_row(), and finally calls get_metrics_python_api() (line 597 of the controller). The new metrics are inserted exactly on that path.
  • driver_carla.py forces this mode anyway. It reads os.environ.get('ROS_VERSION ') with trailing spaces, so it always falls into Python API. controller_carla.py reads the variable correctly, which is why ROS_VERSION=None must be exported.
  • The map is at hand. The controller already has self.carla_map and self.ego_vehicle, so the lane is queried with carla_map.get_waypoint() at the same tick. On the ROS path it would have to be reconstructed from the bag odometry.

πŸ› οΈ Part 2 β€” Modifications

The patch touches two files and is applied by the aplicar_metricas.py script. Line counts were measured applying the script on the actual files: controller_carla.py goes from 847 to 887 lines, and metrics_carla.py from 1777 to 1825. All new code carries the BM_EXTRA_METRICS marker, so grep -n BM_EXTRA_METRICS locates it.

File Function Original anchor line Lines after patch Change
utils/controller_carla.py stop_recording_metrics() 651 (# Guardar) 651–665 Copies the new metrics to the summary if build_json_summary did not include them
utils/controller_carla.py _metrics_headers() 723 ("weather_fog") 739–744 Five new CSV headers
utils/controller_carla.py _sample_row() 766 (w.fog_density, ) 788 *self._bm_lane_info(t) appends the five values to the row
utils/controller_carla.py _bm_lane_info() (new) 770 (before _csv_open) 792–809 Queries the ego lane from the map
utils/metrics_carla.py _bm_extra_metrics() (new) 1612 (before get_metrics_python_api) 1612–1654 Computes the metrics from the CSV
utils/metrics_carla.py get_metrics_python_api() 1723 (setdefault("percentage_completed")) 1767–1771 Calls _bm_extra_metrics() inside a try

driver_carla.py was not touched: its metric list (lines 250–340) is only used in -s mode and could fail with old JSONs missing the new keys. Neither was the GUI Metrics window, which still does not display the new metrics.

New CSV columns

Column Type Meaning
lane_road_idinteger or emptyroad_id of the waypoint under the car
lane_idinteger or emptyCARLA lane_id (the sign indicates the side of the road)
lane_heading_diffdegrees, βˆ’180 to 180Car yaw minus lane yaw; negative = pointing to the left of the lane
on_driving_lane0/11 if the car center is over a Driving-type lane
in_right_lane0/11 if that lane also runs in the car's direction

Lane at each tick: _bm_lane_info() (controller_carla.py)

get_waypoint(..., project_to_road=False) returns None if the car center is not over a driving lane. If it is, the frame counts as right lane when the heading difference with the lane is less than 90Β° (right-hand traffic). With BM_RIGHT_LANE_MODE=rightmost it also requires that there is no other Driving lane to its right.

# --- BM_EXTRA_METRICS: lane of the ego at each step ---
def _bm_lane_info(self, t):
    """[road_id, lane_id, heading_diff, on_driving_lane, in_right_lane] for transform t."""
    try:
        wp = self.carla_map.get_waypoint(
            t.location, project_to_road=False, lane_type=carla.LaneType.Driving
        )
        if wp is None:
            return ["", "", "", 0, 0] # outside a driving lane
        diff = (t.rotation.yaw - wp.transform.rotation.yaw + 180.0) % 360.0 - 180.0
        ok = abs(diff) < 90.0 # same direction as the lane (right-hand traffic)
        if ok and os.environ.get("BM_RIGHT_LANE_MODE", "direction") == "rightmost":
            right = wp.get_right_lane()
            ok = right is None or right.lane_type != carla.LaneType.Driving
        return [wp.road_id, wp.lane_id, round(diff, 2), 1, int(ok)]
    except Exception:
        return ["", "", "", 0, 0]

Metric computation: _bm_extra_metrics() (metrics_carla.py)

Right-lane percentage, with N CSV rows:
right_lane_percentage = (100 / N) Β· Ξ£α΅’ in_right_laneα΅’
Steering zero-crossings per second:

Each steer value (the one applied to the vehicle, get_control().steer) is converted into a sign sα΅’. Values inside the deadband d (0.02 by default, BM_STEER_DEADBAND) have no sign and are discarded. Changes between consecutive remaining signs are counted (K values) and divided by the simulated time T:

sα΅’ = sign(steerα΅’) if |steerα΅’| β‰₯ d ; discarded otherwise
crossings/s = (1/T) Β· Ξ£β‚–β‚Œβ‚‚α΄· [sβ‚– β‰  sₖ₋₁]

Thus, a sequence +0.10 β†’ 0.01 β†’ βˆ’0.10 counts as a single crossing, and noise around 0 on straight sections does not add up.

# -- BM_EXTRA_METRICS --
def _bm_extra_metrics(df, experiment_metrics):
    """Right lane (% of frames) and steering zero-crossings per second."""
    try:
        deadband = float(os.environ.get("BM_STEER_DEADBAND", "0.02"))
    except ValueError:
        deadband = 0.02

    # Right lane: fraction of CSV rows with in_right_lane == 1
    if "in_right_lane" in df.columns and len(df) > 0:
        right = pd.to_numeric(df["in_right_lane"], errors="coerce").fillna(0).to_numpy()
        experiment_metrics["right_lane_percentage"] = float(100.0 * right.mean())
        if "on_driving_lane" in df.columns:
            on = pd.to_numeric(df["on_driving_lane"], errors="coerce").fillna(0).to_numpy()
            experiment_metrics["off_driving_lane_percentage"] = float(100.0 * (1.0 - on.mean()))
    else:
        experiment_metrics["right_lane_percentage"] = None # old CSV without the column

    # Zero-crossings with deadband and hysteresis
    steer = pd.to_numeric(df["steer"], errors="coerce").fillna(0.0).to_numpy()
    signs = np.where(np.abs(steer) >= deadband, np.sign(steer), 0.0)
    firm = signs[signs != 0]
    crossings = int(np.count_nonzero(firm[1:] != firm[:-1])) if len(firm) > 1 else 0
    duration = float(experiment_metrics.get("experiment_total_simulated_time", 0.0) or 0.0)
    experiment_metrics["steering_zero_crossings"] = crossings
    experiment_metrics["steering_zero_crossings_per_second"] = (
        crossings / duration if duration > 0 else 0.0
    )
    experiment_metrics["steering_zero_crossings_deadband"] = deadband
    print("[BM_EXTRA_METRICS] right lane: ... | zero crossings: ...") # console summary
    return experiment_metrics

Call added in get_metrics_python_api(), right after experiment_metrics.setdefault("percentage_completed", 0.0):

# BM_EXTRA_METRICS
try:
    _bm_extra_metrics(df, experiment_metrics)
except Exception as _bm_e:
    print(f"[BM_EXTRA_METRICS] Error computing extra metrics: {_bm_e}")

The aplicar_metricas.py script

  • Backup: saves controller_carla.py.metricas.bak and metrics_carla.py.metricas.bak only the first time.
  • Idempotent: if a file already contains BM_EXTRA_METRICS, it does not touch it.
  • All or nothing: checks all anchors before writing; if one is missing, it does not modify any file.
  • Compilation: compiles each file after the patch and restores the original if it fails.
  • Revert: python3 aplicar_metricas.py --revertir restores the backups.
  • Path: uses ~/BehaviorMetrics/behavior_metrics by default, or the one indicated by BM_DIR.
Environment variableDefaultEffect
BM_STEER_DEADBAND0.02Steering deadband for counting crossings
BM_RIGHT_LANE_MODEdirectionrightmost also requires the rightmost lane
BM_DIR~/BehaviorMetrics/behavior_metricsCode path for the script

πŸš€ Part 3 β€” Step-by-Step to Run the Test

Three terminals are used: Terminal 1 for the simulator, Terminal 2 for Behavior Metrics, and Terminal 3 for checks. Step 0 is done only once.

Step 0 β€” Apply the patch (Terminal 3, once)

With Behavior Metrics closed, copy aplicar_metricas.py to ~/BehaviorMetrics and run it. Running it again does nothing if already applied.

cd ~/BehaviorMetrics && python3 aplicar_metricas.py

Check that in configs/CARLA/pilotnet_town02.yml the Model parameter points to the real .pth and not to 'CAMBIAR/ruta/al/modelo.pth'.

Step 1 β€” Start CARLA 0.9.15 headless (Terminal 1)

The first command removes the containers occupying port 2000, including those from the training project (CARLA 0.9.14).

sudo docker rm -f carla-server carla_montado carla-bm 2>/dev/null
sudo docker run -d --name carla-bm --privileged --gpus all --net=host \
  --env="NVIDIA_VISIBLE_DEVICES=all" --env="NVIDIA_DRIVER_CAPABILITIES=all" \
  carlasim/carla:0.9.15 ./CarlaU4.sh -RenderOffScreen -quality-level=Epic -nosound -world-port=2000

Check that the container is still alive after about 20 s:

sudo docker ps -a --filter name=carla-bm --format '{{.Names}} {{.Status}}'
sudo docker logs --tail 20 carla-bm

If the status is Exited (135) (Signal 7), headless mode did not start. Go back to windowed mode, which does work: add --env="DISPLAY" --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" and remove -RenderOffScreen (you may need xhost +local: beforehand).

Step 2 β€” Confirm that CARLA responds (Terminal 3)
cd ~/BehaviorMetrics/behavior_metrics && source ../venv/bin/activate
python3 -c "import carla; c=carla.Client('localhost',2000); c.set_timeout(20); print('Server', c.get_server_version(), '| map', c.get_world().get_map().name)"

It must print version 0.9.15 and a map. If it times out, wait a few more seconds and repeat.

Step 3 β€” Launch Behavior Metrics (Terminal 2)
cd ~/BehaviorMetrics/behavior_metrics && source ../venv/bin/activate
export ROS_VERSION=None BM_EXTERNAL_CARLA=1 CARLA_ROOT=/tmp
export OBJECT_PATH=$PWD/configs/CARLA/CARLA_launch_files/CARLA_object_files/parked_car_objects.json
# Optional:
export BM_STEER_DEADBAND=0.02 BM_RIGHT_LANE_MODE=direction
python3 driver_carla.py -c configs/CARLA/pilotnet_town02.yml -g

The output must show Controller CARLA ROS VERSION: None and [PilotNet] model_best.pth | architecture=v3 | ... | cuda. The controller waits about 10 s for the map to load and the ego_vehicle to appear.

Step 4 β€” Confirm the weather (Terminal 3)
grep BM_WEATHER /tmp/.worldgen_stdout.log

It must show [BM_WEATHER] Weather applied.

Step 5 β€” Record the experiment (GUI)
  1. In the GUI, start the metrics recording and let the model drive for about 60 s.
  2. Stop the recording. The GUI freezes for a few seconds while computing metrics: this is normal.
  3. In Terminal 2, a line like [BM_EXTRA_METRICS] right lane: ... % | zero crossings: ... in ... s -> .../s must appear.

The results are stored in a YYYYMMDD-HHMMSS folder inside behavior_metrics/ (because Stats.Out is './'), with the CSV, the .meta.json, and the summary .json.

Step 6 β€” Review the new metrics (Terminal 3)
cd ~/BehaviorMetrics/behavior_metrics && source ../venv/bin/activate
D=$(ls -td ./20*/| head -1); echo $D
grep -E "right_lane|off_driving|zero_cross" $D*[0-9].json
python3 -c "import pandas as pd,glob;df=pd.read_csv(glob.glob('$D*[0-9].csv')[0]);print(df[['steer','on_driving_lane','in_right_lane','lane_heading_diff']].describe())"

If steer comes out with standard deviation 0, the motor commands are overriding the steering and robot/interfaces/carla_api_motors.py must be checked. To separate the stopped time after a crash, the following breakdown gives the full metrics, before the first crash, only while moving, and in 5 s segments:

python3 - "$D" <<'EOF'
import sys, glob, pandas as pd, numpy as np
df = pd.read_csv(glob.glob(sys.argv[1] + '*-[0-9].csv')[0])
df['t'] = df.time_stamp - df.time_stamp.iloc[0]
c = df.index[df.collision_impulse > 0.1]
t_col = df.t[c[0]] if len(c) else df.t.iloc[-1]
print(f"First crash: t={t_col:.2f} s" + (f" with {df.collision_with[c[0]]}" if len(c) else " (none)"))
def resumen(nombre, d):
    s = d.steer.to_numpy(); f = np.sign(s[np.abs(s) >= 0.02])
    dur = d.t.iloc[-1] - d.t.iloc[0]
    print(f"{nombre:28s} rows={len(d):5d} dur={dur:5.1f}s right_lane={100*d.in_right_lane.mean():5.1f}% "
          f"off={100*(1-d.on_driving_lane.mean()):5.1f}% steer_mean={s.mean():+.3f} "
          f"crossings/s={np.count_nonzero(f[1:] != f[:-1])/dur if dur>0 else 0:.2f}")
resumen("All", df)
resumen("Before crash", df[df.t < t_col])
resumen("Moving (>0.5 m/s)", df[df.speed_mps > 0.5])
for a, b in [(0,5),(5,10),(10,15),(15,20),(20,30)]:
    resumen(f" t {a}-{b} s", df[(df.t >= a) & (df.t < b)])
EOF
Step 7 β€” Close
  • Terminal 2: Ctrl+C to close Behavior Metrics.
  • Terminal 1: sudo docker stop carla-bm to stop the simulator, or leave it running for another test.
  • To undo the patch (Terminal 3): cd ~/BehaviorMetrics && python3 aplicar_metricas.py --revertir.

πŸ“ˆ Part 4 β€” First Run: PilotNet in Town02

Run 20261020-101251 shows a model that steers smoothly but with a constant left bias, which ends up taking it off the road and crashing around second 20. It lasted 26.95 simulated seconds and produced 2372 rows (about 88 ticks per second); it traveled 66.6 m at an average of 9.1 km/h.

Metric Value Reading
right_lane_percentage53.5%In its lane and in the correct direction a little more than half the time
off_driving_lane_percentage41.9%Outside any driving lane (sidewalk, shoulder, or off the road)
In oncoming lane (58.1 βˆ’ 53.5)β‰ˆ 4.6%Frames with lane_heading_diff near Β±180Β°
steering_zero_crossings8Steering side changes over the whole run
steering_zero_crossings_per_second0.30No zigzagging
steer_mean / medianβˆ’0.084 / βˆ’0.067Left bias; 75% of the values are < 0.008
lane_heading_diff_median (IQR)βˆ’1.76Β° (βˆ’3.6Β° to βˆ’0.24Β°)Inside the lane it always points slightly left

πŸ“Œ Interpretation

Lane. When the car is on the roadway, it is in the correct lane 92% of the time (53.5 / 58.1). The main problem is not invading the oncoming lane but going off the road.

Careful with the 41.9%. It is a percentage of frames, not of distance, and the CSV keeps writing with the car stopped. The collision impulse was active in the last 564 rows (about 6.4 s), so the car was stopped after the crash. If it was outside the lane, that segment alone contributes about 24 points. The time off the road while moving is probably around 18%; the Step 6 breakdown confirms it.

Steering. In CARLA the yaw grows clockwise and a negative steer turns left. The negative mean steering and negative heading inside the lane indicate lateral drift to the left. Without the trajectory, it cannot be ruled out that part of it is a legitimate left curve of the route.

Zero crossings. 0.30/s means the model does not oscillate, but a low value is not good by itself: a biased model that always corrects to the same side also crosses little. It must be read together with the right-lane percentage. In addition, the brain smoothing (0.7 previous + 0.3 new) filters part of the network's oscillation: the metric measures the car, not the raw output.

πŸ§ͺ What the First Run Shows

The two new metrics are correct and capture a real behavior: a smooth but biased controller that drifts left and leaves the road. The next step is to use them as an objective comparison tool across models and seeds, together with the existing cte_media and fuera_pct from the offline/online ranking.

⚠️ Part 5 β€” Limitations and Next Steps

The two new metrics are correct, but collisions and lane invasions in the Python API backend are still inflated by two bugs in the original code:

🐞 Known Bugs That Affect the Metrics

  • Sticky collisions. _last_collision_impulse never returns to 0, so after the first crash every row counts as a collision (564 in this run).
  • Phantom invasions. An empty _last_lane_invasion is saved as an empty field; pandas reads it as NaN, astype(str) turns it into "nan", and every row counts as an invasion (2372 of 2372).
  • Percentage by frames. The CSV keeps writing with the car stopped, so an early crash inflates off_driving_lane_percentage. Versions "while moving" and "up to the first crash" should also be computed.
  • Lane by car center. on_driving_lane looks only at the vehicle center: with half the car outside the lane it still counts as inside.
  • GUI. The Metrics window still does not show the new metrics; they are in the JSON and the console.

πŸ”œ Pending

  • Run the Step 6 breakdown to separate the stopped time after the crash.
  • Patch: reset collision and invasion after recording them and count them per event.
  • Patch: "while moving" and "up to the first crash" metrics.
  • Locate the Metrics window file (grep -rln "suddenness_distance_speed_per_km" --include=*.py ui/ utils/) and add the new metrics.
  • Own reference route for Town02 instead of lap-simple-circuit.

πŸ“Œ WEEK 47 SUMMARY – OCTOBER 20, 2026

🎯 Objective: Add two per-experiment metrics to Behavior Metrics β€” right-lane percentage and steering zero-crossings per second β€” computed in the Python API backend, and validate them with a first PilotNet run in Town02.

πŸ§ͺ Methodologies: Patch applied by aplicar_metricas.py to controller_carla.py and metrics_carla.py (marked BM_EXTRA_METRICS). Five new CSV columns describing the lane at each tick: lane_road_id, lane_id, lane_heading_diff, on_driving_lane, in_right_lane. Right-lane percentage = fraction of rows with in_right_lane = 1. Steering zero-crossings per second = sign changes of steer outside a deadband (0.02 by default) divided by simulated time T. CARLA 0.9.15 headless in Docker, Behavior Metrics with ROS_VERSION=None and BM_EXTERNAL_CARLA=1.

πŸ“ˆ Key Findings: The first run (20261020-101251) shows a smooth but left-biased model: right_lane_percentage = 53.5%, off_driving_lane_percentage = 41.9% (inflated by ~6.4 s stopped after the crash), steering_zero_crossings = 8, steering_zero_crossings_per_second = 0.30 (no zigzagging), mean steer βˆ’0.084 (left bias), median lane_heading_diff βˆ’1.76Β° inside the lane. When on the roadway, the car is in the correct lane 92% of the time. The main failure is going off the road, not invading the oncoming lane.

βœ… Achievements: Two new metrics integrated into the Python API backend and stored in the summary JSON. Patch is idempotent, all-or-nothing, compiles the files after patching, and supports revert. First PilotNet run in Town02 completed and analyzed.

❌ Limitations: Collisions and lane invasions are inflated by two original bugs (sticky collision impulse, phantom invasions). off_driving_lane_percentage is computed by frames and includes stopped time after a crash. Lane detection uses only the vehicle center. The GUI Metrics window does not yet display the new metrics.

πŸ”œ Next Steps: Apply the collision/invasion reset patch and the "while moving" / "up to the first crash" metrics. Run the Step 6 breakdown to separate stopped time. Display the new metrics in the GUI Metrics window. Define an own reference route for Town02. Compare the new lane and steering metrics across seeds and against the Week 46 ranking (cte_media, fuera_pct) to quantify seed variance from a second, independent measurement path.