OK...first of all you are running the risk of breaking your Moodle site if you begin to make changes to the Boost theme.
If anything you would be better off if you downloaded a copy of the Photo theme and use that as your site theme.
Theme_Photo was made so that we developers can see how to make a child theme from Boost. The added beauty of the Photo theme, is that you can add photo images as backgrounds for different pages, but looks and works like Boost.
Going back to your question about the Custom CSS. If it is in boost/style/custom.css stylesheet then have you included that file name into the config.php under...https://github.com/lazydaisy/moodle/blob/master/theme/boost/config.php#L30
$THEME->sheets = [ 'custom' ];
Be careful in what you are doing as it could end up a complete catastrophe!