To solve this, notice that the factorial of any number greater than or equal to 10 will always end in 0, because it will have at least one factor of 10. Thus, only the sum of the factorials of 1 to 9 contributes to the unit digit. Adding them up gives:
\[1! + 2! + 3! + 4! + 5! + 6! + 7! + 8! + 9! = 1 + 2 + 6 + 24 + 120 + 720 + 5040 + 40320 + 362880 = 409113.\]
The unit digit of 409113 is 4.