This script set was originated by Jim McMurray (https://www.jcweather.us/) for his WeatherUnderground History page. My thanks to Jim for letting the script set be adapted for the Saratoga Template set. The data for the page originates with WeatherUnderground.com and was retrieved using a query to WXDailyHistory.asp to return a CSV formatted file with the WUID's station data.
In May, 2019, WeatherUnderground intermittently discontinued the WXDailyHistory.asp page operation, rendering this original script set non-functional. In April, 2020, they also discontinued the graphic chart display so that has been removed in 3.4f version.
I've written a WXDailyHistory.php page to use the WU/TWC API to api.weather.com requests for JSON data, and return the CSV formatted data the script set expects. Due to the limitations with the WU/TWC API, the WXDailyHistory.php script now stores monthly cache files of the returned JSON to speed up processing, and enable full-year queries which do not exist in the WU/TWC API.
The major change from the 3.4c version of the scripts is in WU-History-inc.php to support the new WXDailyHistory.php query instead of directly to the WU website. If you're updating an existing V3.4c installation of the script, you likely need only those two files to restore function to your installation.
Please note that Jim's script also calls a WeatherUnderground URL to display the .gif charts for the period selected. That functionality is based on the legacy WU website, and is likely to be deprecated and removed by WU as their site remodel progresses. At present (March, 2020) the Daily graphs are still working, but is intermittently failing. The Monthly and Yearly graphs are mostly absent, and this is not likely to be fixed in the future as WU spins-down the old infrastructure.
Version 1.00 was the initial release.
Version 1.10 adds additional cache files for day and week data to help keep under the 1500/day, 30/minute rate limits imposed by api.weather.com PwS API. Also, you can include &force=1 to the URL to force a cache reload.
Version 1.20 adds caching for the new daily files wu-YYYYMMDD-{WUID}-{WUunits}.json to help avoid pesky searchbot crawlers from using all your allowed API calls per day.
- For dates prior to yesterday, the script will fetch/cache the data once (since it is a complete day's data).
- For yesterday, it will refresh the file once if it was not a full day of data.
- For today, it will refresh the file every 2.5 minutes.
Version 1.21 adds a bit of code to address an API bug when requesting a day's history for today (in your timezone) and the UTC date is not the same. The current API will return the JSON for yesterday instead of the requested today's data. Logic was added to the DAY processing to request the UTC date instead, which does return the JSON for today (in your timezone). I've opened a problem report on that and will remove the bypass logic when the API is fixed.
Version 1.22 handles bad local/epoch dates in API returns prior to 2018-07-01. It appears that prior to 2018-07, they used a 13-digit epoch date (with 000 miliseconds appended) instead of the customary 10-digit epoch date (in seconds). That caused the JSON "obsTimeLocal" to have odd values like
"tz": "America/Los_Angeles",
"obsTimeUtc": "2018-07-01T06:59:24Z",
"obsTimeLocal": "50467-05-05 22:00:00",
"epoch": 1530428364000,
instead of the customary contents like
"tz": "America/Los_Angeles",
"obsTimeUtc": "2018-07-02T06:59:55Z",
"obsTimeLocal": "2018-07-01 23:59:55",
"epoch": 1530514795,
This version will use the epoch date and tz value to compute a local date (after pruning the epoch date if needed). Old cache files with the JSON will be processed correctly, so no need to reload cache files to get the correct CSV output. My thanks to Holger at http://dl5ark.heliohost.org for spotting the issue leading to the fix.
Version 1.30 Now fixes Radomir's wxwugraphs scripts for month and year displays.
-
Install WXDailyHistory.php and customize for your station.
-
Delete ./wxwugraphs/cache/ entries with station data (since they have the bad formats that are causing the problems). Delete files/directories:
- {WUID}-*.txt and all the
- YYYY-DD directories
Radomir's scripts should fetch/cache what's needed for proper month/year displays by calling WXDailyHistory.php using your existing JSON caches and save the resulting CSV files in the wxwugraphs cache directory. You may have to call the wugraphs.php/wxwugraphs.php page with ?force=1 to get it to regenerate the local cache files needed.
There are still issues with the with the Sun/Solar tab and Wunderground API source, but the others now seem to work fine. Thanks again to Holger for additional testing.
Version 1.31 adds &numericPrecision=decimal to all the WU/TWC API calls to force returned JSON values with decimal values. Also, some additional cURL options to help suppress caching by Akamai of the WU/TWC API returns so data is more current. After installing this version, I recommend you reload your cache files using the At Initial Installation instructions below so the cached JSON will have decimal precision.
- unpack the .zip distribution to the document root of your website (and preserve all the subdirectories/contents)
- customize WU-History-inc.php and WXDailyHistory.php with the particulars. Note you must have a WU/TWC API Key for the script to work.
- upload to your website.
- run the cache preload described below for all the years you have been submitting data to WU (earliest data is 2008)
It's been reported that some unruly search harvester bots are not behaving well with the wxwuhistory.php script and attempting to crawl backwards in the dates, and doing it rapidly and thereby exhausting your allowed 1500 API calls early in the day, so the WXDailyHistory.php script can not fetch data until 24hrs has elapsed. I recommend that you install a block in your .htaccess file like the following example to give the bots a 403-Forbidden instead and thereby preserve your API call quota for useful displays.
SetEnvIfNoCase Referer "^qq829" TOBLOCK=1
SetEnvIfNoCase Referer "^cnzz" TOBLOCK=1
SetEnvIfNoCase ^User-Agent$ .*80legs.* TOBLOCK=1
SetEnvIfNoCase ^User-Agent$ .*Ezooms.* TOBLOCK=1
SetEnvIfNoCase ^User-Agent$ .*Ahrefs.* TOBLOCK=1
SetEnvIfNoCase ^User-Agent$ .*package.* TOBLOCK=1
SetEnvIfNoCase ^User-Agent$ .*scalaj-http.* TOBLOCK=1
SetEnvIfNoCase ^User-Agent$ .*MJ12bot.* TOBLOCK=1
<FilesMatch "(.*)">
Order Allow,Deny
Allow from all
Deny from env=TOBLOCK
# specific 'deny from nnn.nnn.nnn.nnn' entries here
</FilesMatch>
Because WXDailyHistory.php relies on cached JSON data, it's a good idea to preload you cache with historical data manually for all years you have submitted PWS data to WeatherUnderground. The easiest way to do this is via direct URL requests to your website like:
view-source:https://your.website.com/WXDailyHistory.php?&year=2019&graphspan=year&debug=y
Note that you may get a time-out on the page as sometimes the API is a bit laggard in response for a full month's data. No problem, just reload the page until it shows at the bottom something like:
<!-- priorMonthTS='1556693999' ='2019-04-30 23:59:59 PDT' priorYM='201904' -->
<!-- nowYM='201905' tYM='201801' priorYM='201904' fetch='' -->
<!-- fetched 201801 data from ./cache/wu201801-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201802' priorYM='201904' fetch='' -->
<!-- fetched 201802 data from ./cache/wu201802-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201803' priorYM='201904' fetch='' -->
<!-- fetched 201803 data from ./cache/wu201803-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201804' priorYM='201904' fetch='' -->
<!-- fetched 201804 data from ./cache/wu201804-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201805' priorYM='201904' fetch='' -->
<!-- fetched 201805 data from ./cache/wu201805-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201806' priorYM='201904' fetch='' -->
<!-- fetched 201806 data from ./cache/wu201806-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201807' priorYM='201904' fetch='' -->
<!-- fetched 201807 data from ./cache/wu201807-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201808' priorYM='201904' fetch='' -->
<!-- fetched 201808 data from ./cache/wu201808-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201809' priorYM='201904' fetch='' -->
<!-- fetched 201809 data from ./cache/wu201809-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201810' priorYM='201904' fetch='' -->
<!-- fetched 201810 data from ./cache/wu201810-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201811' priorYM='201904' fetch='' -->
<!-- fetched 201811 data from ./cache/wu201811-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201812' priorYM='201904' fetch='' -->
<!-- fetched 201812 data from ./cache/wu201812-KCASARAT1-e.json -->
which indicated all the needed cache files for that year (2018) are present.
For the current year, you will likely see something like:
<!-- priorMonthTS='1556693999' ='2019-04-30 23:59:59 PDT' priorYM='201904' -->
<!-- nowYM='201905' tYM='201901' priorYM='201904' fetch='' -->
<!-- fetched 201901 data from ./cache/wu201901-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201902' priorYM='201904' fetch='' -->
<!-- fetched 201902 data from ./cache/wu201902-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201903' priorYM='201904' fetch='' -->
<!-- fetched 201903 data from ./cache/wu201903-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201904' priorYM='201904' fetch='' -->
<!-- fetched 201904 data from ./cache/wu201904-KCASARAT1-e.json -->
<!-- nowYM='201905' tYM='201905' priorYM='201904' fetch='' -->
<!-- curl fetching 'https://api.weather.com/v2/pws/history/daily?stationId=KCASARAT1&format=json&units=e&startDate=20190501&endDate=20190531&apiKey=-your-api-key-' -->
<!-- HTTP stats: RC=200 dest=104.100.48.232 port=443 (from sce=192.168.1.104)
Times: dns=0.047 conn=0.063 pxfer=0.094 get=0.078 total=0.172 secs -->
<!-- saved 201905 data into ./cache/wu201905-KCASARAT1-e.json -->
where the current month is ALWAYS fetched and cached (since the data for the current day changes)
Note that the API only seems to have data back to 2008 only. Prior years appear to be gone from the WU database, so only go back to (and including 2008) or to the first year after 2008 that you started submitting data to WU.
# ------------------------ settings -----------------------
$WUID = 'KCASARAT1'; // your Wunderground PWS ID
$WCAPIkey = 'specify-for-standalone-use-here'; // use this only for standalone / non-template use
$WCunits = 'e'; // 'e'= US units F,mph,inHg,in,in
//$WCunits = 'm'; // 'm'= metric C,km/h,hPa,mm,cm
//$WCunits = 'h'; // 'h'= UK units C,mph,mb,mm,cm
//$WCunits = 's'; // 's'= SI units C,m/s,hPa,mm,cm
$ourTZ = 'America/Los_Angeles'; // our timezone
$cacheFileDir = './cache/'; // use './' to store in current directory
$refreshSecondsDay = 150; // limit API calls to every 150 seconds (2.5 minutes) for day
$refreshSeconds = 1800; // limit API calls for week/month/year data to every 1/2 hour
# ------------------- end of settings ----------------------Please note that the $WCunits = 'h' or ='s' will work in the returned CSV data, the WU-History-inc.php script is NOT adapted for those units of measure display.
If running in a Saratoga template, the following can be used in Settings.php instead of customizing WXDailyHistory.php
$SITE['WCAPIkey'] = 'specify-your-api-key-here';
$SITE['WCunits'] = 'e'; // ='e' for F,mph,inHg,in; ='m' for C,km/h,hPa,mm
$SITE['WUID'] = 'KCASARAT1'; // your WU PWS ID
$SITE['tz'] = 'America/Los_Angeles'; // timezone
$SITE['cacheFileDir'] = './cache/'; // cache file directory(the same WCAPIkey and WCunits used for the WC-forecast.php script are reused with this script)
- The data returned by the WXDailyHistory.php script is NOT reformatted for number of decimal places. It just presents the data returned in the JSON from the api.weather.com query (so please don't ask me to 'fix' any decimal point issues).
- the api.weather.com API requires a WU/TWC API key which is only available to WU users who have stations directly submitting weather data to WeatherUnderground.
- the 'custom' date range query code is not implemented in V1.00 of WXDailyHistory.php but may appear in subsequent releases.
- use the Issues function on GitHub to report problems with WXDailyHistory.php -- the other parts of the code base I'll try to help, but Jim McMurry is the real authority (it is his code)
Radomir Luza http://pocasi.hovnet.cz/ had last released his scripts in 2011 with little support activity since then (V1.8.0 - 2011/09/05). The link to download the script is no longer operational (in June, 2019). His scripts used the WU WXDailyHistory.asp page for data, which is now deprecated by WeatherUnderground. For those who have a prior installation of his scripts, the WXDailyHistory.php page here seems to be useable with his scripts. To enable it:
- install WXDailyHistory.php in your document root (and configure it so it is working with the WU-History-inc.php set of scripts), then change
http://www.wunderground.com/weatherstation/WXDailyHistory.asp
to
http://your.website.com/WXDailyHistory.php
(replacing 'your.website.com' with your website's hostname) for all instances in the ./wxwugraphs/*.php files. On my copy, I found them in:
-
WUG-inc-day.php
-
WUG-inc-month.php
-
WUG-inc-year.php
-
WUG-pre.php
-
WUG-test.php
-
WUG-ver.php
-
Do the cache preloading on your site as documented above.
-
please note that I did not write, nor directly support Radomir's wxwugraphs/*.php scripts. I do provide support for the WXDailyHistory.php which provides the CSV data needed for his scripts in wudata mode.
