We share our experience in using tensor decomposition as both a supervised and unsupervised methods for detecting malware. We show that tensor decomposition works better with features such as instructions and API calls from disassembled binary than using the more traditional n-grams. Our initial results show a 90% accuracy when used as an unsupervised method, which is higher than the accuracy we observed using a supervised method, such as SVM.