Harvesters

Alpha Vantage

This is a harvestor from the Alpha Vantage data source. for further information you can refer the documentation on the official page: https://www.alphavantage.co/documentation/

class taiyo_utils.harvester.AlphaVantage(retry=5, **kwargs)

This is a harvestor from the Alpha Vantage data source.

Args:

  • Function : The Time Series of your choice of which you need data.
  • start_date : The start date of the data in the YYYY-MM-DD format
  • end_date : The end date of the data in the YYYY-MM-DD format. It is an optional parameter. If not provided it takes system date.
  • retry : The no. of attemps to connect to the API.

Returns:

  • self.data : The dataframe containing the data.
basicPreprocess(self)

This includes basic pre-processing like missing value elimination.

getData(self)

Fetches the Data from the respective endpoints

Returns:

  • Returns a dataframe containing the required data from respective endpoints.

FRED

This is a FRED harvestor for macroeconomic features The apikey can be changed incase the number of API calls finish This site offers a wealth of economic data and information to promote economic education and enhance economic research. For further information you can refer the documentation on the official page: https://fred.nic.cz/documentation/html/

class taiyo_utils.harvester.FRED(**kwargs)

Class to extract data from Fred API.

Arguments:

  • codes List[str]: list of all the codes we need data for.
  • start_date : The start date of the data in the YYYY-MM-DD format
  • end_date : The end date of the data in the YYYY-MM-DD format. It is an optional parameter. If not provided it takes system date.
  • retry : The no. of attemps to connect to the API.

Returns:

  • self.data : The dataframe containing the data.
basicPreprocess(self)

This includes basic pre-processing like missing value elimination.

getData(self)

This function aggregates all the data by calling all Data specified in codes

Returns:

  • Dataframe of the data from FRED API.

Example:

FREDHandle = FREDData(
    codes=["CLVMEURSCAB1GQEU28", "DEXUSEU", "CP0000EU28M086NEST"],
    start_date='2014-02-14',
    end_date='2019-01-28'
    )
FREDHandle.getData()

Finance

This is a harvestor form the Quandl data source. for further information you can refer the documentation on the official page: https://www.alphavantage.co/documentation/

class taiyo_utils.harvester.Financial(**kwargs)

The data is extracted using Quandl api

Arguments:

  • ticker : Series of which data is to be fetched.
  • function: The functions that need to be performed.

Returns:

  • self.data : The dataframe containing the data.
basicPreprocess(self)

This includes basic pre-processing like missing value elimination.

getData(self)

This function aggregates all the data by calling all Data specified in codes

Returns:

  • Dataframe of the aggregate of all the data of the specified code.

Example:

fdata = Financial(
        ticker='AAPL',
        function=['ebit', 'profit', 'beta']
    )
f = fdata.getData()

GDELT

This is a harvestor form the Gdelt data source. It used GCP to fetch data from google bigquery. for further information you can refer the documentation.

class taiyo_utils.harvester.GDELT(**kwargs)

The data is extracted using google bigquery.

Arguments:

  • country : Country Code of Country whose data needs to be fetched.

Returns:

  • self.data : The dataframe containing the data.
basicPreprocess(self)

This includes basic pre-processing like missing value elimination.

getData(self)

This function aggregates all the data by calling all Data specified in codes

Returns:

  • Dataframe of all the data by calling all data specified in codes.

Example:

gd = GDELT(country="US")
f = gd.getData()

InvestPy

This is a harvestor form the InvestPy data source. investpy is a Python package developed in order to retrieve all the available historical data from stocks, funds and ETFs from Investing.com. for further information you can refer the documentation on the official page: https://investpy.readthedocs.io/

class taiyo_utils.harvester.Investing(**kwargs)

The data is extracted using Investing api

Arguments:

  • codes List[str]: list of all the codes we need data for.
  • start_date : The start date of the data in the YYYY-MM-DD format
  • end_date : The end date of the data in the YYYY-MM-DD format. It is an optional parameter. If not provided it takes system date.
  • retry : The no. of attemps to connect to the API.

Returns:

  • self.data : The dataframe containing the data.
basicPreprocess(self)

This includes basic pre-processing like missing value elimination.

getData(self)

This function aggregates all the data by calling all Data specified in codes

Returns:

  • Dataframe of the all the data by calling all data specified in codes.

Example:

IData = Investing(
        codes=['Argentina 3Y'],
        start_date='01/01/2010'
    )
c = IData.getData()

Quandl

This is a harvestor from Quandle data source. Quandl is a marketplace for financial, economic and alternative data. For further information you can refer the documentation on the official page: https://docs.quandl.com/docs

class taiyo_utils.harvester.Quandl(**kwargs)

The data is extracted using Quandl api

Arguments:

  • codes List[str]: list of all the codes we need data for.
  • start_date : The start date of the data in the YYYY-MM-DD format
  • end_date : The end date of the data in the YYYY-MM-DD format. It is an optional parameter. If not provided it takes system date.
  • retry : The no. of attemps to connect to the API.

Returns:

  • self.data : The dataframe containing the data.
basicPreprocess(self)

This includes basic pre-processing like missing value elimination.

getData(self)

This function aggregates all the data by calling all Data specified in codes

Returns:

  • Dataframe of the all the data by calling all data specified in codes.

Example:

qdata = Quandl(
        codes=["USTREASURY/BILLRATES", "USTREASURY/REALYIELD"],
        start_date='01/01/2019',
        end_date="2019-03-10"
    )
c = qdata.getData()

World Bank

class taiyo_utils.harvester.Quandl(**kwargs)

The data is extracted using Quandl api

Arguments:

  • codes List[str]: list of all the codes we need data for.
  • start_date : The start date of the data in the YYYY-MM-DD format
  • end_date : The end date of the data in the YYYY-MM-DD format. It is an optional parameter. If not provided it takes system date.
  • retry : The no. of attemps to connect to the API.

Returns:

  • self.data : The dataframe containing the data.
basicPreprocess(self)

This includes basic pre-processing like missing value elimination.

getData(self)

This function aggregates all the data by calling all Data specified in codes

Returns:

  • Dataframe of the all the data by calling all data specified in codes.

GoogleTrends

This is a harvestor from GoogleTrends. It used pytrends to fetch data from google trends. for further information you can refer the documentation.

class taiyo_utils.harvester.GoogleTrend(**kwargs)

This is a harvestor from the Google trends data source.

Args:

  • country_code : Two letter country abbreviation, For example United States is 'US' More detail available for States/Provinces by specifying additonal abbreviations For example: Alabama would be 'US-AL' For example: England would be 'GB-ENG'
  • search : Keywords to get data for.
  • search_type : What Google property to filter to Example 'news'. Can be images, news, youtube, web search.
  • retry : The no. of attemps to connect to the API.

Returns:

  • self.data : The dataframe containing the data.
basicPreprocess(self)

This includes basic pre-processing like missing value elimination.

getData(self)

Fetches the Data from the respective endpoints Returns:

  • Returns a dataframe containing the required data from respective endpoints.

Example:

gt = GoogleTrend(search=['NLP','machine learning'],country_code=['US','AU'],retry=5)
q = gt.getData()

Economic Policy Uncertainity Index

This is a harvestor from EPU. It fetch data from Policy-Uncertainty website. for further information you can refer the documentation.

class taiyo_utils.harvester.EPU(**kwargs)

This is a harvestor from the Economic Policy Uncertainity Index data source.

Args:

  • country :String(): Country name, For example United States is 'US'
  • retry :int: The no. of attemps to connect to the API.

Returns:

  • self.data : The dataframe containing the data.
basicPreprocess(self)

This includes basic pre-processing like missing value elimination.

getData(self)

Fetches the Data from the respective endpoints Returns:

  • Returns a dataframe containing the required data from respective endpoints.

Example:

a = EPU(country='India',retry=5)
q = a.getData()

Geopolitical Risk Index

This is a harvestor from GRI. It fetch data from Geopolitical Risk Index website. for further information you can refer the documentation.

class taiyo_utils.harvester.GPR(**kwargs)

This is a harvestor from the Geopolitical Risk Index data source.

Args:

  • country :String(): Country name, For example India
  • retry :int: The no. of attemps to connect to the API.

Returns:

  • self.data : The dataframe containing the data.
basicPreprocess(self)

This includes basic pre-processing like missing value elimination.

getData(self)

Fetches the Data from the respective endpoints Returns:

  • Returns a dataframe containing the required data from respective endpoints.

Example:

a = GPR(country='India',retry=5)
q = a.getData()