Environment Configuration

• Run Jupyter in anaconda

(practice_pandas) $ jupyter notebook
[I 2021-11-01 18:52:48.684 LabApp] JupyterLab extension loaded from /Users/joohyunyoon/.pyenv/versions/anaconda3-2021.05/lib/python3.8/site-packages/jupyterlab
...

import pandas as pd
import ssl
from elasticsearch.connection import create_ssl_context
from elasticsearch import Elasticsearch
import urllib3
from decouple import config

ssl_context = create_ssl_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE

ELASTIC_HOST=config('ELASTIC_HOST')
ELASTIC_PORT=config('ELASTIC_PORT')
ELASTIC_USER=config('ELASTIC_USER')
ELASTIC_KEY=config('ELASTIC_KEY')

es = Elasticsearch(hosts=[{'host': 'ELASTIC_HOST', 'port': 'ELASTIC_PORT'}], scheme="http",verify_certs=False, timeout=300, ssl_context=ssl_context, http_auth=("ELASTIC_USER", "ELASTIC_KEY"))
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

######## 2020, 1 year ########
######## There are no MTM data in 2018, 2019 ########

body = {
         "size" : 100,
         "query": {
                 "range":{
                    "TW_COLLECT_DT":{
                        "gte":"2020-01-01T00:00:00.625+09:00",
                        "lte":"2020-12-31T00:00:00.625+09:00" ################
                    }
                }
                 },
    "sort":[{
        "_id":"asc"
    }]
}
        
res = es.search(index = 'd-2k-r-sago-temp', body=body)
data = res['hits']['hits']
# nxt=res["hit"]["hit"][-1]["sort"][0]
total = res['hits']['total']

# print(total)

accident = []
for da in data:
    att_type = da['_source']
    # att_type["POL_NM"]=att_type["SCEN_INFOS"][0]["POL_NM"]
    accident.append(att_type)

# df = pd.DataFrame(accident,dtype=str)
df_10000 = pd.DataFrame(accident)

print(df_10000.head())

>>>
TW_ATT_IP_SEARCH_DATA ACCD_CHARGER_ID TW_ATT_GEOLOCATION  \
0                  None      baenak0717               None   
1                  None         infotel               None

 

• Install Python Decouple

A usuful package that simplifies to access environment variables from whatever environment it is running in.

reference : https://able.bio/rhett/how-to-set-and-get-environment-variables-in-python--274rgt5

$ pip install python-decouple
Collecting python-decouple
...
Successfully installed python-decouple-3.5

 

• Add environment variables.

$ vi .env

ELASTIC_HOST=###.###.##.###
ELASTIC_PORT=####
ELASTIC_USER=
ELASTIC_KEY=