It should be masculine but neuter gender has become acceptable lately. As of September 1, 2009 the Russian Ministry of Education approved the use of either masculine or neuter gender. If you want to please as many people as possible, then use masculine gender because the majority of Russian speakers have known the masculine form to be the correct one.
The word "кофе" doesn't decline (i.e. doesn't change by gender and number). However, some people like to use diminutive forms like "кофеёк" and "кофей" which can be declined. One might say very colloquially to a friend "Не хочешь ли кофейку?" (Would you like some coffee?)
If you try to decline the word "кофе" then it will sound funny and incorrect. But that's being done to achieve a humorous effect. For example, you can use the genitive case and say "нет кофя" (there is no coffee) to show irony or ignorance. Again, such usage is not standard.