Sorry I missed this for almost a year:
Change the following in grade/lib.php
$url = 'edit/letter/index.php?id='.$courseid;
$url = 'edit/custom_letter/index.php?id='.$courseid;
Otherwise the plugin is never called, ever. This has to be done because there is no real way to add editing plugins into the grade book like you can add reports.