Had the same problem (mixing formative scale with regular marked assignments) where the custom scale was throwing off the regular marking. Solution is:
In Site Administration -> General settings
There is an item called "Include Scales in Aggregation". Turn this feature off (on by default). Custom scales are then ignored in the grade calculation.
Hope this helps - drove me crazy for a while.