氣象開放資料平台 潮汐 to google calendar

I wish to make a google calendar for tidal info of certain location, so that I can open google calendar and have the time of high tie of several weekends.
for example: 淡水的 high tie 時間 this weekend.

There are few steps:

Getting data:
different sources comes with different numbers of data and different format.....(why?)
    1. method1: 政府資料開放平臺  it's a JSON file
      1. 潮汐預報-未來1個月潮汐預報https://data.gov.tw/dataset/7038
      2. 潮汐表-明年高低潮時潮高預報(every Oct)https://data.gov.tw/dataset/9205 
      3. this one contains less locations, for 2019 data, it's 42 locations.
    2.  中央氣象局開放資料平臺之資料擷取API: can be json or xml
      1. you need to first get an account and get a key(valid for a year.)
      2. 未來1個月潮汐預報 （no whole year one）
      3. it contain lot of locations,about 210 locations. location table is here: https://opendata.cwb.gov.tw/opendatadoc/MMC/A0021-001.pdf 
      4. I tried the key I found in 政府資料開放平台 some times it works fine...(rdec-key-123-45678-011121314)
Deal with JSON file

這兩種 data的包裝方法完全不同, 資料的密度也完全不同...
我先從氣象局的未來一個月的潮汐開始, 他有210個地點的潮汐資料,但是只有一個月, 所以我必須每個月抓一次資料...而且json file的結構有點混亂

這次使用 google的 Colaboratory 來做這件事, colab是google 的 ubuntu virtual machine, 有完整的 python2 or 3, 預先安裝了很多科學常用的library, 而且還有GPU/TPU可以用, 設計給 machine learning用,尤其是使用 tensorflow...
他的好處是不用去管library的安裝, 而且可以寫注解,單步執行, debug很好用
而且可以輕易的分享,共同建構...
這裏的分享,都可以自己另存新檔, 執行...
這個範例是從中央氣象局開放資料平臺 抓淡水得一個月潮汐資料,並且一層一層的去拆解資料,他真的很多層,很多我不需要的資料...
1_getting 氣象局 tidal_data_in_JSON_getinto_each_data

這個則是把我想要的資料取出,然後輸出CSV file, 這個CSV必須符合google calendar的ㄧ些格式: Import events to Google Calendar - Calendar Help

3_write 氣象局 one month tidal data to csv

測試ok後, 我只想在google calendar上show weekend的資料...

4_write 氣象局 one month_tidal_data_to_csv_weekend_only

辛苦的做了這麼久, 只能show一個月, 以後每個月都要run一次, 每個月的csv還會跟上個月有重複的問題, 這很麻煩, 因為google calendar可以用CSV批次匯入,但是沒有辦法用CSV做批次的刪除....


所以我轉向政府資料開放平臺
 https://data.gov.tw/dataset/9205 
潮汐表-明年高低潮時潮高預報
每年10月提供明年高低潮時潮高預報，地點、潮位、時間、潮高 about 25-26MB, 但是資料的格式非常不同,只是地點只有42個,每個都有一個ID...

一年份潮汐表內容的拆解,取淡水 ID 1102

只截取今天之後的weekend 放進csv










留言

張貼留言

這個網誌中的熱門文章

Meshtastic 03 adding GPS on TTGO lora32 V2

圖片
 I wish to add GPS on my running meshtastic client on TTGO lora32. I have a GPS module outputs with nema at 9600bps. In the config app, in the position sector, you  can find the redefine of PGS pins. I firstly use IO00 as GPS_RX_pin. But it does not work. I guess the web config use 0 as none-config default. Then I want to know what is the default pin for GPS in meshtastic for TTGO Lora 32, I use the 2.4.x firmware for V2.1-1.6. Checked the git, there is no default gps pin for this hardware, so I changed the gps to mcu pin to 34, leave other pins stuff as 0 and do the config .  And it works. there is a new flasher, which still has LILYGO T-Lora V2.1-1.6 as board selection, with latest 2.5.xx firmware maintain 34,0,0 setting is no longer working. after few try, I decided to set to 34/32/25, where 32/25 are free pin on my board. I also reference to  LILYGO Meshtastic T-Beam V1.2 ESP32 LoRaGPS WiFi , it has a gps on 34/12 with 34/12/0 the setting is still working
閱讀完整內容

GeoSetter how to add geotag on to pictures, and export to google earth as kmz

圖片
I use gps info of my cell phone when I take pictures, so that I can find where these pics were taken after a long voyage. I also wish to export these pics to google earth or google map, which can be kml or kmz. https://geosetter.de/en/download-en/ Geo setter is a very powerful one. Some times my phone was not able to add geotag info on the the picture, since it does not have location info at that time(indoor picture or gps was not available). Of course you can edit the pic, but geo sitter provide a better interface, you can pic a location on map and then assign the location to the photo that you selected. much easier than deal with coordinates. I notes the steps as following. After install geosetter, select the folder where you store your images. photos with a blue location dot means these pics contain geotag. select one pic with no blue dot, and go to top menu "map", select "show overview map"  a google map will show up with goe location of pictures...
閱讀完整內容

AIS0. understanding AIS NMEA 0183: How it's coded

For my boat tracking system, I found out that using AIS format seems to be a good idea, at least I don't have to worry about how to present boat data, since there are already plenty things for do this on mobile phone or PC. Here is what I read: http://catb.org/gpsd/AIVDM.html Looking into AIS NMEA0183, it has much more application then I think. Such as MOB device, epir, aid to navigation device...etc, all are included. it use the first few char to indicate the function: Table 1. AIS talker IDs !AB NMEA 4.0 Base AIS station !AD MMEA 4.0 Dependent AIS Base Station !AI Mobile AIS station !AN NMEA 4.0 Aid to Navigation AIS station !AR NMEA 4.0 AIS Receiving Station !AS NMEA 4.0 Limited Base Station !AT NMEA 4.0 AIS Transmitting Station !AX NMEA 4.0 Repeater AIS station !BS Base AIS station (deprecated in NMEA 4.0) !SA NMEA 4.0 Physical Shore AIS Station It can also be used between device on boat for message communication, AIVDO The...
閱讀完整內容