Comparing automated text classification methods

Online social media drive the growth of unstructured text data. Many marketing applications require structuring this data at scales non-accessible to human coding, e.g., to detect communication shifts in sentiment or other researcher-defined content categories. Several methods have been proposed to automatically classify unstructured text. This paper compares the performance of ten such approaches (five lexicon-based, five machine learning algorithms) across 41 social media datasets covering major social media platforms, various sample sizes, and languages. So far, marketing research relies predominantly on support vector machines (SVM) and Linguistic Inquiry and Word Count (LIWC). Across all tasks we study, either random forest (RF) or naive Bayes (NB) performs best in terms of correctly uncovering human intuition. In particular, RF exhibits consistently high performance for three-class sentiment, NB for small samples sizes. SVM never outperform the remaining methods. All lexicon-based approaches, LIWC in particular, perform poorly compared with machine learning. In some applications, accuracies only slightly exceed chance. Since additional considerations of text classification choice are also in favor of NB and RF, our results suggest that marketing research can benefit from considering these alternatives.     

An empirical comparison of machine learning methods for text-based sentiment analysis of online consumer reviews

The amount of digital text-based consumer review data has increased dramatically and there exist many machine learning approaches for automated text-based sentiment analysis. Marketing researchers have employed various methods for analyzing text reviews but lack a comprehensive comparison of their performance to guide method selection in future applications. We focus on the fundamental relationship between a consumer’s overall empirical evaluation, and the text-based explanation of their evaluation. We study the empirical tradeoff between predictive and diagnostic abilities, in applying various methods to estimate this fundamental relationship. We incorporate methods previously employed in the marketing literature, and methods that are so far less common in the marketing literature. For generalizability, we analyze 25,241 products in nine product categories, and 260,489 reviews across five review platforms. We find that neural network-based machine learning methods, in particular pre-trained versions, offer the most accurate predictions, while topic models such as Latent Dirichlet Allocation offer deeper diagnostics. However, neural network models are not suited for diagnostic purposes and topic models are ill equipped for making predictions. Consequently, future selection of methods to process text reviews is likely to be based on analysts’ goals of prediction versus diagnostics.     

User preference mining based on fine-grained sentiment analysis

User preference mining is an application of data mining that attracts increasing attention. Although most of the existing user preference mining methods achieved significant performance improvement, the sentiment tendencies of users were seldom considered. This paper proposes fine-grained sentiment analysis for preference mining. The powerful feature representation capabilities of deep neural networks have significantly improved the performance of fine-grained sentiment analysis. But two main challenges remain when using deep neural network models: incomplete user feature extraction and insufficient interaction. In response, a pre-training language model is employed to encode user features to fully explore potential interests of users, a linguistic knowledge model is introduced to assist the encoding, a multi-scale convolution neural network is adopted to capture text features at different scales and fully utilize the text information, and the fine-grained sentiment analysis task is modeled as a sequence labeling problem to explore the sentiment polarity of user evaluation. Experiments on a user review data set are used to verify the new approach. Experimental results of precision, recall rate and F1-value show that the proposed approach performs better, and is more effective than baseline models. For example, the F1-value is increased by 4.27% compared to the best performing baseline model. Findings have important implications for research and practice.     

Seeing the wood for the trees: How machine learning can help firms in identifying relevant electronic word-of-mouth in social media

The increasing volume of firm-related conversations on social media has made it considerably more difficult for marketers to track and analyse electronic word-of-mouth (eWOM) about brands, products or services. Firms often use sentiment analysis to identify relevant eWOM that requires a response to consequently engage in webcare. In this paper, we show that sentiment analysis of any kind might not be ideal for this purpose, because it relies on the questionable assumption that only negative eWOM is response-worthy and it is not able to infer meaning from text. We propose and test an approach based on supervised machine learning that first decides whether eWOM is relevant for the brand to respond, and then—based on a categorization of seven different types of eWOM (e.g., question, complaint)—classifies three customer satisfaction dimensions. Using a dataset of approximately 60,000 Facebook comments and 11,000 tweets about 16 different brands in eight different industries, we test and compare the efficacy of various sentiment analysis, dictionary-based and machine learning techniques to detect relevant eWOM. In doing so, this study identifies response-worthy eWOM based on the content instead of its expressed sentiment. The results indicate that these machine learning techniques achieve considerably higher accuracy in detecting relevant eWOM on social media compared to any kind of sentiment analysis. Moreover, it is shown that industry-specific classifiers can further improve this process and that algorithms are applicable across different social networks.     

Ambidexterity and the evolution of knowledge management initiatives

Current debates on organizational learning distinguish two distinct and mutually exclusive learning modes: exploration and exploitation. The paper deals with the concept of ambidextrous routines in knowledge management (KM) initiatives. The case study-based findings put this understanding into perspective, as the authors have identified KM initiatives that concurrently facilitate exploration and exploitation. The paper elaborates the characteristics of ambidextrous KM initiatives at micro-level; firms use ambidextrous KM practices to create a learning context, defined rather by guidelines and methods than by a definite purpose. The clear separation of KM initiatives' purpose (aim) and their embedded learning routines and methods enables them to be used ambidextrously. Furthermore, this analysis indicates that ambidextrous KM initiatives follow a path characterized by an increasing variety of purposes but a decreasing variety of underlying structures. Consequently, firms create a learning context that can be activated when necessary in ways required either in an exploratory and/or in an exploitative mode.     

Learning from error: The influence of error incident characteristics

This study investigates the influence of error incident characteristics on organizational learning among operators in the chemical process industry. The study asks operators to describe recently occurred error incidents at time 0 (n = 87), followed up by measurements for learning 6 weeks later (n = 48). Organizations learn more from error incidents with more severe consequences. Severity of consequences relates positively to learning. When consequences are more severe, communication about an error is higher. Communication is subsequently related to learning. Error incidents without imminent negative consequences, however, can also be a platform for learning. This research recommends attention towards the promotion of learning from conditions that do not necessarily encourage employees to learn.     

Making new products go viral and succeed

Why are some new product introductions more viral and successful than others? This research integrates theories of interpersonal communication and consumer learning to explore this question. Analyzing a unique data set of millions of consumer word-of-mouth transmissions (eWOM) on social media regarding 345 new automobile products introduced during 2008–2015, we find that more innovative products generate more eWOM volume but surprisingly less positive sentiment. These effects vary in magnitude across eWOM channels. However, the use of rich-content communication, pre-announcement, and cobranding strengthens (weakens) the positive (negative) effect of product innovativeness on eWOM volume (sentiment). The results further indicate that eWOM sentiment is a stronger predictor of new product success than eWOM volume. Experimental results reveal more insights into how product innovativeness influences eWOM metrics in several product categories and shed light on the role of excitement and perceived risk as mechanisms underlying these effects. The research offers useful implications for firms to design effective viral marketing campaigns to enhance new product success.     

Social Media's Impact on the Consumer Mindset: When to Use Which Sentiment Extraction Tool?

User-generated content provides many opportunities for managers and researchers, but insights are hindered by a lack of consensus on how to extract brand-relevant valence and volume. Marketing studies use different sentiment extraction tools (SETs) based on social media volume, top-down language dictionaries and bottom-up machine learning approaches. This paper compares the explanatory and forecasting power of these methods over several years for daily customer mindset metrics obtained from survey data. For 48 brands in diverse industries, vector autoregressive models show that volume metrics explain the most for brand awareness and purchase intent, while bottom-up SETs excel at explaining brand impression, satisfaction and recommendation. Systematic differences yield contingent advice: the most nuanced version of bottom-up SETs (SVM with Neutral) performs best for the search goods for all consumer mind-set metrics but Purchase Intent for which Volume metrics work best. For experienced goods, Volume outperforms SVM with neutral. As processing time and costs increase when moving from volume to top-down to bottom-up sentiment extraction tools, these conditional findings can help managers decide when more detailed analytics are worth the investment.     

Antecedents of Retweeting in a (Political) Marketing Context

Word of mouth disseminates across Twitter by means of retweeting; however, the antecedents of retweeting have not received much attention. We used the chi‐square automatic interaction detection (CHAID) decision tree predictive method (Kass, 1980 ) with readily available Twitter data, and manually coded sentiment and content data, to identify why some tweets are more likely to be retweeted than others in a (political) marketing context. The analysis includes four CHAID models: (1) using message structure variables only, (2) source variables only, (3) message content and sentiment variables only, and (4) a combined model using source, message structure, message content, and sentiment variables. The aggregated predictive model correctly classified retweeting behavior with a 76.7% success rate. Retweeting tends to occur when the originator has a high number of Twitter followers and the sentiment of the tweet is negative, contradicting previous research (East, Hammond, & Wright, 2007 ; Wu, 2013 ) but concurring with others (Hennig‐Thurau, Wiertz, & Feldhaus, 2014 ). Additionally, particular types of tweet content are associated with high levels of retweeting, in particular those tweets including fear appeals or expressing support for others, while others are associated with very low levels of retweeting, such as those mentioning the sender's personal life. Managerial implications and research directions are presented. We make a methodological contribution by illustrating how CHAID predictive modeling can be used for Twitter data analysis and a theoretical contribution by providing insights into why retweeting occurs in a (political) marketing context.     

The Validity of Investor Sentiment Proxies

Behavioral finance research relies on proxies for unobservable phenomena. Different proxies for the same underlying phenomena should be correlated (formal proof of this proposition is presented in this letter). This letter examines proxies for an unobservable variable, sentiment. We utilize a well‐known methodology to construct text‐based sentiment proxies and compare these with metrics from Baker and Wurgler. We find that they are not correlated. At least one, but perhaps all, of these are not valid proxies of sentiment.     

Narcissism and learning from entrepreneurial failure

Failure of a prior business provides an opportunity for an entrepreneur to learn in the subsequent entrepreneurial endeavor, but learning from failure is not guaranteed. Why do some entrepreneurs learn less from failure than others? In this study, we propose that a narcissistic personality can create cognitive and motivational obstacles to learning. We further posit that the inhibiting effect of narcissism will be more salient when the costs of failure, especially social costs, are higher. Our analysis with a survey sample of startups provides the initial empirical evidence about the negative impact of narcissism on learning from entrepreneurial failure. The study adds to research on learning from failure and narcissism in entrepreneurship.     

The conditions for successful knowledge transfer in development‐aid training programs

The shortage of qualified human capital is a major impediment to development. In the field of international development cooperation, training programs (TPs) have been widely employed to enhance the capacity of workforces in developing countries. This paper investigates the conditions in which these programs can contribute not only to individual human resource development but also to organization‐level reform and innovation in developing countries. The methods were regression analyses of training monitoring records as well as follow‐up e‐mail interviews with former participants of information and communication technology TPs sponsored by the Japanese International Cooperation Agency. The research reveals that bilateral communication between training participants and the home organizations during the training plays a key role in increasing the probability of successful organization‐level transfer of individual‐level learning, irrespective of the original level of organization's absorptive capacity. The researchers examine the differences in transfer factors between the development aid context in their research and the paradigm case of the business organization found in much of the transfer literature.     

Microfoundations for learning within international joint ventures

IJV research highlights the importance of learning in international joint ventures (IJVs) but has not indicated how to achieve it. We combine organizational learning and internationalization process research within a microfoundations framework to understand learning in IJVs. We study a Samsung–Tesco IJV that successfully learned retail practice from one partner and applied it in a South Korean context known by the other. The managers used many learning processes, not just experiential learning emphasized in international business research, and used many more knowledge sources than assumed in prior research, including the IJV partners’ other subsidiaries. To build absorptive capacity, IJVs need appropriate microfoundations at individual, process and structural levels, and coherent interlinkages between them, especially by having IJV managers’ with extensive experience and orientation to learn who are given structural and process autonomy to invest in learning.     

When consumers can return digital products: Influence of firm- and consumer-induced communication on the returns and profitability of news articles

The news industry is being massively disrupted by the digital distribution of news. Consequently, publishers have revised their business models and integrated pay-per-article options. To reduce pre-purchase uncertainty, consumers can use information from firm-induced (e.g., newsletters), or consumer-induced communication (e.g., likes). These communication activities avoid purchases with poor fit but also increase customer expectations. Consequently, their effect on sales, returns, and profitability is unclear. For digital products, these effects are even less clear because product quality is difficult to evaluate pre-purchase, and products can be returned at almost no cost, even after consumption. In this study, we investigate the effects of firm- and consumer-induced communication on digital returns in the context of news articles on a major online platform (Blendle). We rely on a multi-equation model to quantify the effect of firm- and consumer-induced communication activities (i.e., newsletter promotions sent out by the platform and consumer likes from readers) on sales and returns and calculate their profitability impact. Our results show that newsletters decrease returns but do not significantly affect sales. In contrast, consumer likes have a twofold effect by increasing sales and decreasing returns. A simulation shows that both newsletters and likes increase profitability and that likes have a higher potential. Our results offer much needed guidance for online aggregators of digital products (e.g., audiobooks, e-books or news articles), as well as for online publishers based on pay-per-unit business models.     

‘We (don’t) know how you feel’ – a comparative study of automated vs. manual analysis of social media conversations

Abstract

The ever-growing volume of brand-related conversations on social media platforms has captivated the attention of academics and practitioners, as the analysis of those conversations promises to offer unparalleled insight into consumers’ emotions. This article takes a step back from the hype, and investigates the vulnerabilities related to the analysis of social media data concerning consumers’ sentiment. A review of the literature indicates that the form, focus, source and context of the communication may negatively impact on the analyst’s ability to identify sentiment polarity and emotional state. Likewise, the selection of analytical tool, the creation of codes, and the classification of the data, adversely affect the researcher’s ability to accurately assess the sentiment expressed in a social media conversation. Our study of Twitter conversations about coffee shows low levels of agreement between manual and automated analysis, which is of grave concern given the popularity of the latter in consumer research.     

In bot we trust: A new methodology of chatbot performance measures

Chatbots are used frequently in business to facilitate various processes, particularly those related to customer service and personalization. In this article, we propose novel methods of tracking human-chatbot interactions and measuring chatbot performance that take into consideration ethical concerns, particularly trust. Our proposed methodology links neuroscientific methods, text mining, and machine learning. We argue that trust is the focal point of successful human-chatbot interaction and assess how trust as a relevant category is being redefined with the advent of deep learning supported chatbots. We propose a novel method of analyzing the content of messages produced in human-chatbot interactions, using the Condor Tribefinder system we developed for text mining that is based on a machine learning classification engine. Our results will help build better social bots for interaction in business or commercial environments.     

Does culture affect sentiments expressed in cruise tours’ eWOM?

The study aims to examine the sentiment differences in the content of cruise tour online reviews across North Americans and Europeans, as representing the two largest cruise markets. Dictionary-based sentiment analysis has been carried out on 1127 reviews on guided tours retrieved from TripAdvisor. The results indicate significant differences in the sentiment score of the reviews, with North Americans’ texts being more emotionally charged than the European ones. In addition, North Americans’ reviews conveyed a more positive affect and had a more subjective and intimate tone, while those written by Europeans contained a smaller amount of sentiment-bearing words and their tone was more objective. The study’s contribution lies in (i) providing evidence for the influence of culture on electronic word-of-mouth communication in terms of varying sentiment expression, (ii) demonstrating the effectiveness of sentiment analysis for recognizing cultural differences and (iii) enhancing the current understanding of cruisers’ tour experience.     

Modeling preference evolution in discrete choice models: A Bayesian state-space approach

We develop discrete choice models that account for parameter driven preference dynamics. Choice model parameters may change over time because of shifting market conditions or due to changes in attribute levels over time or because of consumer learning. In this paper we show how such preference evolution can be modeled using hierarchial Bayesian state space models of discrete choice. The main feature of our approach is that it allows for the simultaneous incorporation of multiple sources of preference and choice dynamics. We show how the state space approach can include state dependence, unobserved heterogeneity, and more importantly, temporal variability in preferences using a correlated sequence of population distributions. The proposed model is very general and nests commonly used choice models in the literature as special cases. We use Markov chain monte carlo methods for estimating model parameters and apply our methodology to a scanner data set containing household brand choices over an eight-year period. Our analysis indicates that preferences exhibit significant variation over the time-span of the data and that incorporating time-variation in parameters is crucial for appropriate inferences regarding the magnitude and evolution of choice elasticities. We also find that models that ignore time variation in parameters can yield misleading inferences about the impact of causal variables. This paper is based on the first author's doctoral dissertation.     

Relationship learning: A conduit for internationalization

Internationalization opportunities can emerge through inter-organizational sharing, yet research on why and how organizations learn through relationship interactions is underdeveloped. We explore how learning in supplier-customer relationships contributes to organizational offerings through the knowledge development process. We identify relationship learning as an organizational dynamic capability by thematic analysis of qualitative longitudinal data from large as well as small and medium-sized organizations. Our case study of organizations demonstrates that nurturing personal relationships and paying attention to customer communication is core in knowledge sharing. Customer input is valuable in solution offerings, strengthening mutual work, and growth in internationalization within an existing relationship or in new ones. The results endorse that the knowledge development processes and commitments transpire at both ends of the relationship. The findings provide practical managerial implications for ensuring the development of open and transparent communication conduits in relationships. The process of providing a solution that addresses customers' needs must begin with understanding their work, issues, and the intended jobs they will perform.     

Team‐based learning and training transfer: a case study of training for the implementation of enterprise resources planning software

While traditional models of training such as behavioral modeling (BMT) have been found to enhance training transfer, research suggests that more active learning strategies such as error management (EMT) and team‐based learning (TBL) may be more effective. This paper analyzes BMT, EMT and TBL strategies to train employees on new enterprise resources planning (ERP) software and discusses which training leads to successful procedural and declarative knowledge transfer, knowledge retention and application, and tangible business outcomes. TBL was predicted to be the most effective training type, as it models several components needed to use ERP software in the actual job setting. Overall and procedural knowledge as well as knowledge application scores improved most for TBL participants, while declarative knowledge improved the most in the EMT condition. During training, all conditions showed significant improvement in knowledge application; however, the TBL condition showed the highest knowledge application gains. This paper discusses the elements of TBL that support its use as an effective strategy to increase knowledge transfer in an organizational context.