Ah never mind, I think I figured it out.
I think some of the online documentation is a bit misleading. It looks like the new "one piece" (mod/attendance) module (filename: mod_attendance_moodle24_2013070404.zip) is compatible with both Moodle 2.3.x and 2.4.x. However in some places, it mentioned that for Moodle 2.3.x, we should use the "old" version which would create a /mod/attforblock folder (filename: danmarsden-moodle_mod_attforblock_old-1a0168d.zip). These two are not compatible. After I removed the old one (attforblock) and just keep the new one (/mod/attendance), it works perfectly. Just a quick FYI.