Publication

Abstract : Individuals utilize online networking sites like Facebook and Twitter to express their interests, opinions or reviews. The users used English language as their medium for communication in earlier days. Despite the fact that content can be written in Unicode characters now, people find it easier to communicate by mixing two or more languages together or lean toward writing their native language in Roman script. These types of data are called code-mixed text. While processing such social-media data, recognizing the language of the text is an important task. In this work, we have developed a system for word-level language identification on code-mixed social media text. The work is accomplished for Tamil-English and Malayalam-English code-mixed Facebook comments. The methodology used for the system is a novel approach which is implemented based on features obtained from character-based embedding technique with the context information and uses a machine learning based classifier, Support Vector Machine for training and testing. An accuracy of 93% was obtained for Malayalam-English and 95% for Tamil-English code-mixed text.