Weeks 8-11: Influence of image size

In order to get closer to real-time prediction, we’ve been testing how image size influence both execution time and performance. We can experiment with different image sizes tuning boxsize parameter, defined in CPMs configuration file. Before testing different box sizes, I have analyzed how the sample that goes through the model changes until the pose estimation is reached and how it depends on boxsize.

1. Original image is resized according to boxsize.
2. Human detector is fed with the resized image.
3. Human detector outputs a heatmap eight times smaller than its input because of pooling and convolutional layers stride.
4. The heatmap is resized back to the size of the image that fed the human detector.
5. With the human coordinates obtained from the heatmap, a squared region of size boxsize is cropped.
6. Human box is fed to the pose estimator.
7. Pose estimator outputs joint coordinates over an image eight times smaller than its input.
8. Finally, these joint coordinates are transformed to fit full size image.

Originally, boxsize was equal to 384. These are the results obtained with different box sizes:

Boxsize	Human detection (s)	Pose estimation (s)	Total (s)
384	19.43	13.08	32.51
192	4.81	3.04	7.65
128	2.31	1.43	3.74
92	1.21	-	-

As it can be seen, when we reduce size, we get a very significant speed-up, but predictions become less accurate or even non-existent. A good trade-off is reached with 192x192 boxsize: predictions are 4x times faster and they still being pretty accurate.