DJIA_Stocks_Web-Scrapping.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stock Price Prediction Dow Jones 30 companies (DJIA) - (2000-01-01 to 2020-18-05)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The Dow Jones Industrial Average (DJIA) is a famous and widely referenced stock market indicator. It started in the year 1885 and is the second oldest active market index. \n",
    "\n",
    "#### Currently, 30 different companies make up the Dow Jones Industrial Average. It covers almost all major US markets except for transportation and utilities. It initially started with just 12 companies. \n",
    "\n",
    "#### The Dow Jones Industrial Average is very different. The DJIA is a \"price-weighted\" index, meaning that the companies with higher stock prices have greater weight in the index's calculation. But the DJIA isn't just the sum of the stock prices of the 30 stocks: It's a scaled average, meaning that the index's value is adjusted to account for historic stock splits, dividends, and other changes to the individual stocks over time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The companies/ stocks that make up the DJIA are as follows:\n",
    "\n",
    "#### 3M (NYSE: MMM),       American Express (NYSE: AXP),       Apple (NASDAQ: AAPL), \n",
    "#### Boeing (NYSE: BA),       Caterpillar (NYSE: CAT),        Chevron (NYSE: CVX),         \n",
    "#### Cisco (NASDAQ: CSCO),          Coca-Cola (NYSE: KO),            The Walt Disney Company (NYSE:DIS),           \n",
    "#### DowDuPont (NYSE:DD),           ExxonMobil (NYSE:XOM),          General Electric (NYSE:GE),           \n",
    "#### Goldman Sachs (NYSE:GS),            The Home Depot (NYSE:HD),             IBM (NYSE:IBM),             \n",
    "#### Intel (NASDAQ:INTC),         Johnson & Johnson (NYSE:JNJ),          JPMorgan Chase (NYSE:JPM),            \n",
    "#### McDonald's (NYSE:MCD), Merck (NYSE:MRK),             Microsoft (NASDAQ:MSFT),               \n",
    "#### Nike (NYSE:NKE),             Pfizer (NYSE:PFE),               Procter & Gamble (NYSE:PG),            \n",
    "#### Travelers Companies (NYSE:TRV),                United Technologies (NYSE:RTX),               UnitedHealth (NYSE:UNH),  \n",
    "#### Verizon (NYSE:VZ),                Visa (NYSE:V),            Wal-Mart (NYSE:WMT)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Let us web scrap the above stocks from google."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Importing required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import pandas_datareader as pdr\n",
    "import datetime"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "start_date = datetime.datetime(2000, 1, 1)\n",
    "end_date   = datetime.datetime(2020, 1, 1)\n",
    "\n",
    "start_date_str = str(start_date.date())\n",
    "end_date_str   = str(end_date.date())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "DJIA_Stocks = ['MMM', 'AXP', 'AAPL', 'BA', 'CAT', 'CVX', 'CSCO', 'KO', 'DIS', 'DD', 'XOM', 'GE', 'GS', 'HD', 'IBM', \n",
    "               'INTC', 'JNJ', 'JPM', 'MCD', 'MRK', 'MSFT', 'NKE', 'PFE', 'PG', 'TRV', 'UTX', 'UNH', 'VZ', 'V', 'WMT']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MMM_2000-01-01_to_2020-01-01.csv\n",
      "(4505, 12)\n",
      "AXP_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "AAPL_2000-01-01_to_2020-01-01.csv\n",
      "(4585, 12)\n",
      "BA_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "CAT_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "CVX_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "CSCO_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "KO_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "DIS_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "DD_2000-01-01_to_2020-01-01.csv\n",
      "(4445, 12)\n",
      "XOM_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "GE_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "GS_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "HD_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "IBM_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "INTC_2000-01-01_to_2020-01-01.csv\n",
      "(4585, 12)\n",
      "JNJ_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "JPM_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "MCD_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "MRK_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "MSFT_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "NKE_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "PFE_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "PG_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "TRV_2000-01-01_to_2020-01-01.csv\n",
      "(231, 12)\n",
      "UTX_2000-01-01_to_2020-01-01.csv\n",
      "(4586, 12)\n",
      "UNH_2000-01-01_to_2020-01-01.csv\n"
     ]
    }
   ],
   "source": [
    "csv_path_to_save = 'D:/Datasets/DJIA_Stocks_Web-Scrapping'\n",
    "\n",
    "for ticker in DJIA_Stocks:\n",
    "    file_name = ticker + '_' + start_date_str + '_to_' + end_date_str + '.csv'\n",
    "    print(file_name)\n",
    "    data = pdr.DataReader(ticker, 'quandl', start_date, end_date, api_key = 'WmshVudqyoxhq_jLxndh')\n",
    "    print(data.shape)\n",
    "    data.to_csv(csv_path_to_save + file_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}