氣象開放資料平台 潮汐 to google calendar

I wish to make a google calendar for tidal info of certain location, so that I can open google calendar and have the time of high tie of several weekends.
for example: 淡水的 high tie 時間 this weekend.

There are few steps:

Getting data:
different sources comes with different numbers of data and different format.....(why?)
    1. method1: 政府資料開放平臺  it's a JSON file
      1. 潮汐預報-未來1個月潮汐預報https://data.gov.tw/dataset/7038
      2. 潮汐表-明年高低潮時潮高預報(every Oct)https://data.gov.tw/dataset/9205 
      3. this one contains less locations, for 2019 data, it's 42 locations.
    2.  中央氣象局開放資料平臺之資料擷取API: can be json or xml
      1. you need to first get an account and get a key(valid for a year.)
      2. 未來1個月潮汐預報 （no whole year one）
      3. it contain lot of locations,about 210 locations. location table is here: https://opendata.cwb.gov.tw/opendatadoc/MMC/A0021-001.pdf 
      4. I tried the key I found in 政府資料開放平台 some times it works fine...(rdec-key-123-45678-011121314)
Deal with JSON file

這兩種 data的包裝方法完全不同, 資料的密度也完全不同...
我先從氣象局的未來一個月的潮汐開始, 他有210個地點的潮汐資料,但是只有一個月, 所以我必須每個月抓一次資料...而且json file的結構有點混亂

這次使用 google的 Colaboratory 來做這件事, colab是google 的 ubuntu virtual machine, 有完整的 python2 or 3, 預先安裝了很多科學常用的library, 而且還有GPU/TPU可以用, 設計給 machine learning用,尤其是使用 tensorflow...
他的好處是不用去管library的安裝, 而且可以寫注解,單步執行, debug很好用
而且可以輕易的分享,共同建構...
這裏的分享,都可以自己另存新檔, 執行...
這個範例是從中央氣象局開放資料平臺 抓淡水得一個月潮汐資料,並且一層一層的去拆解資料,他真的很多層,很多我不需要的資料...
1_getting 氣象局 tidal_data_in_JSON_getinto_each_data

這個則是把我想要的資料取出,然後輸出CSV file, 這個CSV必須符合google calendar的ㄧ些格式: Import events to Google Calendar - Calendar Help

3_write 氣象局 one month tidal data to csv

測試ok後, 我只想在google calendar上show weekend的資料...

4_write 氣象局 one month_tidal_data_to_csv_weekend_only

辛苦的做了這麼久, 只能show一個月, 以後每個月都要run一次, 每個月的csv還會跟上個月有重複的問題, 這很麻煩, 因為google calendar可以用CSV批次匯入,但是沒有辦法用CSV做批次的刪除....


所以我轉向政府資料開放平臺
 https://data.gov.tw/dataset/9205 
潮汐表-明年高低潮時潮高預報
每年10月提供明年高低潮時潮高預報，地點、潮位、時間、潮高 about 25-26MB, 但是資料的格式非常不同,只是地點只有42個,每個都有一個ID...

一年份潮汐表內容的拆解,取淡水 ID 1102

只截取今天之後的weekend 放進csv










留言

張貼留言

這個網誌中的熱門文章

Heltec ESP32+OLED+Lora, hardware testing

圖片
you can easily find some good boards from china, but most of them lack of accurate document... Hrere is an example; https://hackaday.io/project/26991-esp32-board-wifi-lora-32 the board looks good with lora and a 128x64 OLED, but the IIC and reset connection is important, since pin can be muxing easily, so every board could be different. pinMode(16,OUTPUT);  digitalWrite(16, LOW); // set GPIO16 low to reset OLED  delay(50);  digitalWrite(16, HIGH); // while OLED is running, must set GPIO16 to high  and, if needed, edit your Wire.begin() to Wire.begin(4,15); (4 = SDA 15 SCL)  i tested it with the Adafruit SSD1306 and the Acrobotics SSD1306 Librarys  both work.  Display size is 128x64 There are discussions on these boards on TＴＮ as a LORA node..or maybee single ch gateway? Heltec has a git :  https://github.com/Heltec-Aaron-Lee And there is another copy call TTGO https://de.aliexpress.com/item/TTGO-LORA32-868-915-Mhz-SX1276-ESP32-Oled-display-Bluetooth-WIFI
閱讀完整內容

micro SD card for ESP32, on lolin32 with OLED and heltec 32 lora oled

圖片
I have purchased 2 micro SD card holder for my esp project. First I would like to test with heltec esp32 lora oled module. this module uses: sck=05 MISO=19 MOSI=27 SS(lora)=18, then I assigned 21 for sd SS and there is a lib in esp32-arduino, which use SS default at 5 I took the example and use SD.begin(21), but I alway get the error: entry 0x40078a9c Card Mount Failed But I do saw once that it read out the sd card type and volume. Not sure it's the issue of lora sd card swithing, or it just can't work. The other esp32 card I have is an wemos esp32 oled, it did not leave me too much pins, I need to find out how to connect sd to it. But I have wemos D1 mini esp8266 on hand, and connect it as:  * The WeMos Micro SD Shield uses:  * D5, D6, D7, D8, 3V3 and G  *  * The shield uses SPI bus pins:  * D5 = CLK  * D6 = MISO  * D7 = MOSI  * D8 = CS It's not working with the example in the arduino IDE 1.8.5, which marked SD(esp8266), but it w
閱讀完整內容

Install Network Time Protocol(NTP) on BeagleBone with Angstrom linux and set local time zone

Referenced from : http://cwraig.id.au/?p=513 1. Update your Package List opkg update 2. Get the ntp Package opkg install ntp ntpdate Update 2014/07/03 for the latest beaglebone image(2013-06-20) with kernel 3.8.12, after this installation,  system will add to service list, so only need to change time zone 3. Now we need to stop the ntpd service so we can request a manual update /etc/init.d/ntpd stop manual update ntpdate pool.ntp.org type in date, you should see the date is changing to current date and time, but the time zone is not corrected. The time zone info is contained in /etc/localtime , use this to remove it: rm /etc/localtime before link up our time zone, look into /usr/share/zoneinfo folder for your timezone. I did not find taiwan in asia, not plan to use hong_kong, but there is still: /usr/share/zoneinfo/Etc/GMT-8 now we make the link with ln command ln -s /usr/share/zoneinfo/Etc/GMT-8 /etc/localtime now type date again to check, should ha
閱讀完整內容