Abstract

This study explores the impact of artificial intelligence (AI) on stock price prediction in India, focusing on the application of Long Short-Term Memory (LSTM) networks and hybrid models. By analyzing data from the Bombay Stock Exchange (BSE) including stock prices, trading volumes, and sentiment data from social media and news, the study compares AI-based models with traditional methods. The research aims to determine whether AI models offer superior predictive accuracy using RMSE, MAE, and MAPE. The findings suggest that AI models, particularly LSTM and hybrid approaches, outperform traditional models in forecasting accuracy, offering a significant advantage for stock market prediction in India. The study concludes AI models provide a significant improvement in stock price prediction accuracy compared to traditional methods. This suggests the adoption of AI models in stock price prediction by Indian financial institutions and regulatory bodies. This study provides valuable insights for investors, financial institutions, and policymakers looking to integrate AI into market forecasting strategies.

1. Introduction

Advancements in AI techniques such as machine learning, deep learning, and natural language processing have revolutionized financial modeling. These methods improve predictive accuracy by capturing complex patterns in historical stock data, sentiment analysis, and macroeconomic indicators ¹. Traditional financial theories argue that stock prices reflect all available information ². AI challenges this notion by leveraging unstructured data, news, and social media to uncover actionable insights. Indian stock markets are characterized by high volatility, driven by macroeconomic policies, global cues, and retail investor sentiment. This provides an ideal testbed for AI models. India has seen increased digitization of financial markets, making historical data accessible for training AI models ³. Studies have demonstrated the effectiveness of AI in stock prediction using LSTM (Long Short-Term Memory) for time-series forecasting or sentiment analysis from news articles ⁴. Research in India focuses on incorporating local market conditions, regulatory changes, and cultural factors into predictive models ⁵. AI aids retail investors and portfolio managers in devising better trading strategies by analyzing large datasets. Regulators and policymakers can use AI-driven insights to identify systemic risks and ensure market stability ⁶.

The field of stock price prediction has always been a challenging and dynamic area of research. With the advent of artificial intelligence (AI), traditional methods of forecasting stock prices have been supplemented by advanced machine learning algorithms, neural networks, and natural language processing tools. These technologies have revolutionized the financial market by enhancing the accuracy of predictions and automating complex data analysis processes. Stock price prediction is vital for investors, policymakers, and analysts to make informed decisions about investments, portfolio management, and market interventions ⁷. The volatility of stock markets, driven by factors such as economic indicators, company performance, geopolitical events, and investor sentiment, makes accurate predictions a highly sought-after capability. Machine learning and deep learning have proven effective in analyzing large datasets, detecting patterns, and making data-driven predictions ⁸. In India, where the stock market is influenced by unique socio-economic and regulatory factors, AI offers the ability to process diverse and complex data sources, such as historical stock prices, macroeconomic indicators, corporate announcements, and news articles and social media sentiment. The Indian stock market is one of the most dynamic in emerging economies. The use of AI in this market has gained traction due to increased availability of financial data through digital platforms, technological advancements in computing power, and growth in algorithmic trading. While AI presents significant opportunities for improving stock price prediction, challenges remain. These include data quality issues, ethical concerns, and the need for interpretability in AI models. Additionally, regulatory frameworks in India are evolving to address the implications of AI-driven decision-making in financial markets ^{9 10 11}.

The volatility and complexity of the Indian stock market pose significant challenges for investors and financial analysts in predicting stock prices with accuracy. Traditional statistical and econometric methods fail to capture the intricate, non-linear patterns in financial data influenced by dynamic market conditions, macroeconomic variables, and investor sentiment. The advent of artificial intelligence (AI) provides promising tools for addressing these challenges. However, the application of AI in stock price prediction is not without its limitations. Issues such as data quality, overfitting, lack of model interpretability, and regulatory uncertainties impede its widespread adoption ^{7 10}. Furthermore, the Indian stock market, characterized by high retail participation, diverse investment behaviors, and a mix of emerging and mature industries, adds layers of complexity that AI models must navigate. Despite the proliferation of AI-based solutions, there is limited empirical evidence on their effectiveness and practical implications in the Indian context ^{12 13}. Indian stock markets are highly sensitive to global economic cues, domestic policies, and retail investor behavior, leading to significant price fluctuations ³. While large datasets are available, issues like missing data, noise, and unstructured formats hinder the effective application of AI ⁵. The Securities and Exchange Board of India (SEBI) imposes strict regulations that may impact the use of algorithmic and AI-driven trading strategies ⁶. Long Short-Term Memory (LSTM) networks require extensive tuning and large datasets to generalize well. Their performance in dynamic environments like India needs validation ⁴. While global studies demonstrate AI's potential for stock prediction, the Indian context with its unique blend of market dynamics and investor behaviour lacks substantial empirical research. This gap limits stakeholders' understanding of how AI can be effectively deployed in Indian markets. Thus, the problem lies in understanding the impact, opportunities, and constraints of AI-driven approaches in predicting stock prices in India. This research seeks to address how effective are AI-based models in forecasting stock prices in the Indian market compared to traditional methods?

Stock price prediction is a critical component of financial decision-making, guiding investors, portfolio managers, and policymakers. Traditional approaches like statistical modeling and econometric techniques struggle with the vast, dynamic, and nonlinear nature of financial data. AI provides advanced computational tools to improve accuracy by analyzing complex patterns in historical and real-time data. Machine learning (ML), deep learning (DL), and natural language processing (NLP) have demonstrated superior performance in financial forecasting. They can incorporate a wide range of data sources, including news sentiment, social media trends, and macroeconomic indicators, providing a holistic view of market dynamics. The Indian market offers a unique testing ground due to its high volatility, diverse investor base, and influence of both global and local factors. The Indian stock market is among the most prominent in emerging economies. However, the literature on AI applications in stock price prediction remains limited for India. This gap calls for a deeper understanding of how AI can enhance prediction models in this specific context, particularly given challenges like retail investor dominance and regulatory constraints ^{14 15}.

AI has emerged as a transformative force in the financial sector, offering innovative tools to analyze large, complex datasets and improve prediction accuracy. The success of AI applications in global financial markets motivates the exploration of their effectiveness in the Indian context, which presents unique challenges such as high retail participation, diverse sectors, and socio-economic influences. Despite global advancements, the application of AI in stock price prediction remains underexplored in India. Most existing studies focus on traditional techniques or global markets, leaving a significant research gap in understanding the impact and feasibility of AI in the Indian stock market. Stock price prediction plays a vital role in supporting informed investment decisions, risk management, and economic growth. With India’s stock market contributing significantly to the economy and attracting global investors, enhancing prediction methods using AI has the potential to strengthen market efficiency and investor confidence ^{15 16}.

The study is specific to the Bombay Stock Exchange (BSE) Sensex. It also considers diverse industry sectors and their stock behaviors to analyze how AI adapts to the complexities of the Indian market. The analysis considers historical data from at least the past decade to ensure robustness and reliability of results, with a focus on the evolution of AI's role in stock price prediction.

The aim of the study is to analyze the effectiveness and implications of artificial intelligence (AI) in predicting stock prices in the Indian stock market (BSE-Sensex), by comparing AI-based models with traditional forecasting methods.

2. Literature Review

The growing interest in leveraging Artificial Intelligence (AI) for stock price prediction stems from its ability to analyze vast amounts of data and uncover hidden patterns. This literature review synthesizes global and India-specific studies to provide insights into the evolution, methodologies, and challenges of AI in stock price prediction. Fama ² explained the Efficient Market Hypothesis (EMH), forming the theoretical foundation for AI applications that challenge EMH by extracting hidden patterns. Samuel ¹⁷ stated one of the earliest works on machine learning, laying groundwork for AI methods in dynamic stock markets. AI models such as Support Vector Machines (SVM), Random Forest (RF), and Neural Networks (NN) have proven effective in forecasting stock prices by capturing non-linear and complex relationships in data. Chen et al. ¹² explored LSTM models, providing a comparative framework for emerging markets like India. Patel et al. ⁷ utilized a combination of ML techniques to predict stock indices, finding that hybrid models improved accuracy compared to standalone methods. Fischer & Krauss ⁴ demonstrated the efficacy of LSTM networks in time-series forecasting, with implications for Indian markets. Jain & Dandapat ⁸ applied deep learning techniques like Long Short-Term Memory (LSTM) networks in the Indian stock market, demonstrating their ability to capture temporal dependencies in stock prices. Sentiment analysis using NLP tools has enhanced stock price prediction by incorporating data from news articles, social media, and other textual sources. Bollen et al. ⁹ showed that social media sentiment could predict stock market movements, a concept increasingly applied in India. Ghosh & Mukherjee ¹⁰ investigated the integration of sentiment analysis with ML models, highlighting its relevance for predicting Indian stock market trends. Research comparing AI-driven approaches with traditional econometric methods, such as ARIMA and GARCH, underscores AI's superiority in handling high-dimensional and non-linear data. Mehta & Sharma ¹⁸ investigated ANN-based approaches and their predictive accuracy for Indian equities. Sahoo & Pradhan ⁵ examined ML models in Indian stock price prediction, highlighting data quality issues. Chakrabarti & Kumar ¹⁵ found that AI models outperformed conventional methods in terms of prediction accuracy and responsiveness to market fluctuations in India. Mitra & Bera ¹⁶ emphasized that AI models are particularly adept at processing large datasets and adapting to evolving market conditions. Despite AI's potential, challenges such as data quality, overfitting, and lack of model interpretability persist. Agarwal & Bhuvaneshwari ¹⁴ highlighted data availability and preprocessing as critical barriers in the Indian context, given the diversity of market conditions and investment behaviors. Ethical and regulatory concerns, as discussed by Agrawal et al. ¹³, further complicate the adoption of AI in financial markets. The Indian stock market is uniquely positioned to benefit from AI due to its high retail participation and increasing digitalization. Sebastiao & Godinho ¹¹ argued that emerging markets like India could leverage AI to improve market efficiency and attract global investors. Chakrabarti & Kumar ¹⁵ stressed the importance of algorithmic transparency and the need for regulatory frameworks to ensure the ethical use of AI in financial forecasting.

A study by Mehtab and Sen ²⁵ employed Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks to predict the NIFTY 50 index's opening prices. Their findings indicated that a univariate encoder-decoder convolutional LSTM model, utilizing data from the previous two weeks, achieved superior accuracy. In contrast, a univariate CNN model with one week of prior data was noted for its execution speed. Further research by Sen et al. ²⁰ applied an LSTM model to historical stock prices from 180 stocks across 18 sectors listed on the National Stock Exchange (NSE) of India. The study demonstrated the model's effectiveness in accurate stock price prediction and its utility in guiding investment decisions, offering insights into sector profitability and volatility. Darapaneni et al. ¹⁹ integrated sentiment analysis with deep learning models to forecast stock movements in the Indian market. They utilized LSTM models with historical price data and Random Forest models incorporating sentiment scores from news articles, alongside macroeconomic indicators like gold and oil prices. The combined approach improved prediction accuracy for stocks such as Reliance, HDFC Bank, TCS, and SBI. A more recent study by Attaluri et al. ²¹ leveraged 30 years of historical data from Indian national banks, combining it with sentiment analysis from tweets and reputable financial sources. The research employed advanced deep learning models, including multivariate multi-step LSTM and Facebook Prophet with LightGBM, optimized through Optuna, to forecast stock prices, acknowledging the significant influence of news sentiment on market fluctuations. Chandanshive and Ansurkar ²² conducted a comparative study of AI techniques—Artificial Neural Networks (ANNs), Support Vector Machines (SVMs), and Random Forests (RFs)—for predicting the Indian stock market. Analyzing historical data from the NSE Nifty 50 index, they found that ANNs outperformed SVMs and RFs in terms of prediction accuracy and directional accuracy. Similarly, Barua et al. ²³ compared deep learning models for stock price prediction in the Indian market. Their analysis underscored the effectiveness of LSTM models over other deep learning approaches, highlighting the importance of model selection in achieving accurate forecasts.

While AI models claim superiority over traditional econometric models, there is insufficient empirical evidence comparing their performance in the Indian context. Studies rarely assess the trade-offs between the computational complexity of AI models and the interpretability of traditional methods. The cost-benefit analysis of implementing AI-based solutions in terms of infrastructure and expertise remains largely unexplored.

3. Data and Methodology

The research adopts an exploratory and comparative design to analyze the effectiveness of AI-based stock price prediction models. The stock price data has been collected from Bombay Stock Exchange (BSE). Sentiment data has been gathered from online news sources, social media, and financial reports. Sentiment scores have been extracted using Natural Language Processing (NLP) tools, which analyze text data for positive, negative, or neutral sentiment. This sentiment data is served as an additional feature to improve AI model predictions, particularly in volatile or news-driven market conditions. Relative Strength Index (RSI) has been included to provide additional features for model training. It is calculated using historical price data obtained from the stock exchange. The span of the dataset is for 10 years (from 2014 to 2024), providing a diverse set of market conditions for model training and testing. In the course of analysis, descriptive statistics, correlation analysis, ADF unit root test, multiple regression analysis (ARIMA-LSTM Hybrid), residual analysis, our of sample forecasting performance analysis, and cross validation test results have been used.

The ADF test extends the Dickey-Fuller test by including lagged differences of the dependent variable to address autocorrelation (Dickey & Fuller, 1979). The general form of the ADF equation is:

Δy_t =α+β_t+ γy_t−1 +∑ δ_i Δy_t−I +ϵ_t

Where y_t: The time series being tested.

Δy_t=y_t−y_t−1: First difference of the series.

α: Constant term (for testing trend).

β_t: Time trend.

γ: Coefficient to test for the presence of a unit root (γ=0 implies a unit root).

δ_i: Coefficients for lagged differences.

ϵ_t: Error term.

The PP test adjusts the Dickey-Fuller statistic to account for serial correlation and heteroscedasticity in the error terms without adding lagged differences (Phillips, & Perron, 1988). The equation resembles the standard Dickey-Fuller test: Δy_t=α+β_t+γy_t−1+ϵ_t

The distinction lies in how the test modifies the t-statistics of γ using non-parametric methods to account for autocorrelation and heteroscedasticity.

In the context of analyzing the impact of AI on stock price prediction in India, a multiple regression model can be used to estimate the relationship between stock prices and AI-based indices, macroeconomic indicators, and market sentiment ²⁴.

Y_t=β₀+β₁X_1t+β₂X_2t+⋯+β_kX_kt+ϵ_t

Y_t: Dependent variable (stock price or return).

X_1t, X_2t,…, X_kt: Independent variables (AI sentiment scores, market indices, trading volume).

β₀: Intercept.

β₁, β2,…, β_k: Coefficients of predictors.

ϵ_t: Error term.

4. Empirical Results and Analysis

Empirical analysis of AI's impact on stock price prediction involves data-driven methods, econometric tests, and machine learning models.

Table 1 show that the mean value of stock prices indicates the average price over the sample period. Standard deviation shows the level of volatility. The higher the standard deviation, the more volatile the stock is. The sentiment score represents the sentiment polarity (positive or negative) derived from news or social media, with values closer to 1 indicating positive sentiment and closer to 0 indicating neutral/negative sentiment. RSI (Relative Strength Index) is a momentum oscillator that measures the speed and change of price movements. RSI values above 70 indicate overbought conditions, and below 30 indicate oversold conditions.

Correlation analysis measures the strength and direction of the linear relationship between two or more variables. In stock price prediction, it is used to understand how different variables (stock prices, trading volume, sentiment scores, technical indicators) relate to each other.

Table 2 shows that the correlation coefficient of 0.35 suggests a weak positive relationship between stock price and trading volume. This indicates that, as stock prices increase, trading volumes tend to rise, but the relationship is not strong. A correlation of 0.22 indicates a weak positive relationship between stock prices and sentiment scores, suggesting that positive sentiment has a slight tendency to influence stock price increases. The correlation coefficient of -0.45 indicates a moderate negative relationship, meaning that as the RSI increases (indicating overbought conditions), stock prices tend to decrease, aligning with technical analysis principles. A low positive correlation (0.14) shows that sentiment and RSI are weakly correlated, suggesting they are somewhat independent in influencing stock prices.

Stationarity is a key assumption for many time series models. A time series is stationary if its statistical properties (mean, variance, autocorrelation) do not change over time. In stock price prediction, non-stationary data can lead to unreliable model results. The Augmented Dickey-Fuller (ADF) test is used to check for stationarity at the level. The null hypothesis of the ADF test is that the series has a unit root (the series is non-stationary), and the alternative hypothesis is that the series is stationary.

Table 3 shows that the p-value of stock price is less than 0.05, so we reject the null hypothesis. This suggests that the stock price series is stationary at level, meaning that it can be used directly for time series models. The p-value of trading volume is less than 0.05, indicating that there is an evidence to reject the null hypothesis. It suggests that the trading volume series is stationary. The p-value of sentiment score is below 0.05, so we reject the null hypothesis and conclude that the sentiment score series is stationary at level, meaning it can be used directly for modeling. The p-value of RSI is well below 0.05, meaning we reject the null hypothesis and conclude that the RSI series is stationary at level, indicating that it is suitable for use in predictive models without further transformation.

Table 3A also shows that the p-value of stock price is less than 0.05, so we reject the null hypothesis. This suggests that the all the variables are stationary at level, meaning that it can be used directly for time series models.

Table 4 demonstrates that the constant term shows the baseline level of stock price when all independent variables are zero. The coefficient (0.872) of stock price (lag 1) suggests that the previous day's stock price has a significant positive effect on the current day's price, with a high t-statistic (16.44) and low p-value (<0.05), indicating statistical significance. The coefficient (0.015) of trading volume suggests a positive relationship with stock price, but the p-value (0.062) is slightly above the significance threshold (0.05), suggesting weaker evidence of a relationship. The positive coefficient (5.154) indicates that positive sentiment derived from news or social media has a significant impact on stock prices, as the p-value (0.027) is statistically significant. The negative coefficient (-0.423) implies that higher RSI values (overbought conditions) are associated with a decrease in stock price, with a highly significant p-value (0.000).

Table 5 reveals that Jarque-Bera test statistic (3.71) with a p-value of 0.157 suggests that the residuals are normally distributed, as the p-value is greater than 0.05. LM test for autocorrelation in the residuals shows that the p-value of 0.058 is just above 0.05, indicating slight evidence of autocorrelation, but it’s not conclusive. ARCH Test (Heteroscedasticity) for changing variance over time (volatility clustering), the p-value of 0.061 suggests no strong evidence of heteroscedasticity, though it is borderline.

Table 6 reveals that the lower the RMSE, the better the model’s prediction accuracy. The AI-based LSTM model (45.34) and the hybrid model (43.50) outperform ARIMA (50.21). The LSTM and hybrid models have lower MAE values, indicating they have smaller average prediction errors than the ARIMA model. A percentage-based metric with lower MAPE indicates a better forecasting accuracy. The hybrid model has the lowest MAPE (1.5%), indicating superior performance in forecasting stock prices. R² indicates the proportion of variance in the dependent variable (stock price) explained by the model. The LSTM-based and hybrid models show higher R-squared values (0.96 and 0.97), indicating better fit compared to ARIMA (0.92).

Cross-validation results (Table 7) show that the ARIMA + LSTM hybrid model consistently provides the lowest RMSE, MAE, and MAPE, indicating its superiority in terms of prediction accuracy. LSTM also outperforms ARIMA, especially in terms of prediction accuracy over multiple folds.

5. Conclusions

The correlation analysis shows that multiple factors, such as trading volume, sentiment scores, and RSI, can influence stock prices. AI-based models can integrate these diverse variables to improve prediction accuracy, especially when combined with other factors like historical stock prices. Stationarity tests reveal that all variables are ready for direct modeling, ensuring more reliable results in time series forecasting models and AI-based methods. AI-based models, particularly LSTM and hybrid models, demonstrate superior predictive accuracy over traditional time series models like ARIMA. These models can better capture complex relationships between stock prices, sentiment, technical indicators, and trading volume, leading to more accurate stock price predictions. The hybrid AI model consistently outperforms other models in forecasting accuracy, making it a strong candidate for stock price prediction in India. Cross-validation results further highlight the robustness of AI models, particularly in real-world applications where data is noisy and complex. The AI-based models not only provide accurate forecasts but also meet important diagnostic checks, suggesting that they are robust and well-suited for stock price prediction in India. Overall, the study confirms that AI-based models, particularly hybrid models that incorporate various financial and sentiment variables hold significant potential for improving stock price prediction accuracy in the Indian market.

The analysis shows that AI models consistently outperform traditional models like ARIMA in terms of forecasting accuracy. By promoting AI, India’s stock market could benefit from better predictive accuracy and more informed trading decisions. Develop guidelines or frameworks to allow and promote the use of AI models in market forecasting. This includes creating a legal and regulatory environment that fosters innovation while ensuring transparency and fairness in AI algorithms. The analysis shows that hybrid models have the potential to outperform both traditional econometric models and standalone machine learning models by leveraging the strengths of both approaches. Hybrid models can handle a wider range of market dynamics, including non-linear relationships and volatile market conditions. Encourage the adoption of hybrid models in stock price prediction by offering grants, incentives, or funding for research on AI applications in financial markets. Financial institutions and research bodies could be incentivized to explore and test hybrid models to develop more robust and adaptive forecasting tools. The use of AI-based models for stock price prediction in India offers significant potential for improving the accuracy and efficiency of financial forecasting. By creating an enabling environment for AI-driven innovations, India can position itself at the forefront of AI applications in financial markets.

References