Search Members Help

» Welcome Guest
[ Log In :: Register ]

Pages: (2) < [1] 2 >

[ Track this topic :: Email this topic :: Print this topic ]

reply to topic new topic new poll
Topic: XML cache data, Cache data and logs via XML< Next Oldest | Next Newest >
GeoRockers Offline
Caching Maniac




Group: Members
Posts: 167
Joined: Feb. 2002
Posted: Aug. 26 2002,6:48 am QUOTE

The cache data and logs are now accessible via XML! :D

The script has been functional since May, but procrastination and an accident delayed the announcement. :(


  • You may use this cache data for personal and non-commercial use.

  • You may redistribute this cache data for personal and non-commercial use as long as original source of cache data is declared (source=NaviCache).

  • If this data is used on another cache server, then permission is granted to NaviCache for accessing of other caching data at the cache server (fair use in kind).




The script can be accessed at:

http://www.navicache.com/cgi-bin/db/XMLSearchResults.pl


The supported arguments are:

  city=
  state=
  country=
  difficulty=
  terrain=
  NewOrModifiedSince=YYYY-MM-DD HH:MM:SS
  AllCacheData=0 or 1 (default=0)
  logs=0 or 1 (default=0)
  LogsOnly=0 or 1 (default=0)
  LogStats=0 or 1 (default=0)
  LogStatsOnly=0 or 1 (default=0)
  gzip=0 or 1 (default=0)

format for the DATE attribute is 'YYYY-MM-DD'              
format for the DATE_TIME attribute is 'YYYY-MM-DD HH:MM:SS'
latitude and longitude are in hours (WGS-84 datum)


To retrieve an XML listing of all caches which have been added or modified since May 1, 2002:

Code Sample

http://www.navicache.com/cgi-bin/db/XMLSearchResults.pl?NewOrModifiedSince=2002-05-01



To retrieve an XML listing of all caches which have been added or modified since May 1, 2002, along with the cache description and hints:

Code Sample

http://www.navicache.com/cgi-bin/db/XMLSearchResults.pl?NewOrModifiedSince=2002-05-01&AllCacheData=1


To retrieve an XML listing of all caches which have been added or modified since May 1, 2002, along with the cache description, hints, and logs:

Code Sample

http://www.navicache.com/cgi-bin/db/XMLSearchResults.pl?NewOrModifiedSince=2002-05-01&AllCacheData=1&logs=1



George

[2004-02-12] updated arguments list


Edited by GeoRockers on Feb. 12 2004,9:21 pm

--------------
A 1/1 via the trail around the hill?
Nah... that's too easy!  Let's go up and over!
Why follow a trail when we can blaze our own? :p

We *make* mountains out of mole hills!
Back to top
Profile PM 
Scout Offline
Cache Master




Group: Members
Posts: 721
Joined: June 2001
    Posted: Aug. 26 2002,3:25 pm QUOTE

Quote (GeoRockers @ Aug. 26 2002,12:48 am)
The cache data and logs are now accessible via XML!

Nice feature.

--------------
Scout  ( http://GPSgames.org )
Back to top
Profile PM 
m.zielinski Offline
Navicacher




Group: Members
Posts: 1
Joined: Jan. 2003
    Posted: Jan. 19 2003,4:11 pm QUOTE

I really would like to use this feature but i dont have any programm which understands this special xml-codes..
Couldnt you implement it like that: http://www.topografix.com/gpx_resources.asp

That format is e.g. also understood by gpsbabel my favourite converter...
Back to top
Profile PM 
PC Medic Offline
Cache Master




Group: Super Administrators
Posts: 1450
Joined: Dec. 2001
    Posted: Jan. 19 2003,4:36 pm QUOTE

I can not give an exact date yet, but this was actually one of the topics of discussion we (the site owners) just had this past week. Looks like GPX format got the vote and should be implemented here on navicache.com .

--------------
'PC Medic'
Back to top
Profile PM AOL YIM 
infosponge Offline
Navicacher




Group: Members
Posts: 12
Joined: June 2003
    Posted: June 20 2003,1:41 pm QUOTE

Are there more arguments I can pass in, namely zip code or lat/lon + radius?

We're trying to match up the queries on Tampabaygeocaching.com with what we can pull from geocaching.com as a GPX file, but if all I can specify is Tampa, FL ... I get 5 caches.   There's more out there...I want people who visit the site to be aware of what's out there so they use both sites.

What I'm going to do now is pull all of the Florida caches and skip any that are outside of a specfic lat/lon "box", but that's wasteful.
Back to top
Profile PM WEB 
PC Medic Offline
Cache Master




Group: Super Administrators
Posts: 1450
Joined: Dec. 2001
    Posted: June 20 2003,3:42 pm QUOTE

All three.

If you would like to PM or email me your tel # I will call and work out with you the info you need.


--------------
'PC Medic'
Back to top
Profile PM AOL YIM 
Icenians Offline
Navicacher




Group: Members
Posts: 36
Joined: Sep. 2002
    Posted: June 26 2003,9:14 am QUOTE

Thanks for this. A great feature.
Back to top
Profile PM 
PC Medic Offline
Cache Master




Group: Super Administrators
Posts: 1450
Joined: Dec. 2001
    Posted: June 27 2003,6:55 pm QUOTE

We aim to please. Keep an eye out for GPX soon

--------------
'PC Medic'
Back to top
Profile PM AOL YIM 
infosponge Offline
Navicacher




Group: Members
Posts: 12
Joined: June 2003
    Posted: July 08 2003,9:25 pm QUOTE

For some reason, this cache:

http://www.navicache.com/cgi-bin/db/displaycache2.pl?CacheID=1812

has a hidden on date of 08-Jul-2003, but when I get the XML file, it has 08-Jun-2003 instead.  I have a "newest caches" feature on my site and it seems like newly listed Navicaches never show up because the dates are off by a month in many cases.    This also happened with another cache that was listed 05-Jul-2003, it shows up as 05-Jun-2003 in the XML file.

For now, I'll check and manually patch this up on my end, but I wanted to make you aware of the problem.
Back to top
Profile PM WEB 
Quinn Offline
Caching Madman




Group: Members
Posts: 1146
Joined: June 2001
    Posted: July 08 2003,9:46 pm QUOTE

Thanks sponge. I think we had this problem once before and it may be caused by editing the cache at this end.
We will look into it and see whats going on here  ???


--------------
"I Cache...Therefore I am!"
    Quinn Stone
Back to top
Profile PM ICQ 
PC Medic Offline
Cache Master




Group: Super Administrators
Posts: 1450
Joined: Dec. 2001
    Posted: July 08 2003,9:46 pm QUOTE

The cache appears correctly in the database and in checking the XML generation script I find nothing there that would cause this. I will have another more thorough look.

--------------
'PC Medic'
Back to top
Profile PM AOL YIM 
PC Medic Offline
Cache Master




Group: Super Administrators
Posts: 1450
Joined: Dec. 2001
    Posted: July 09 2003,7:19 am QUOTE

Quote (Quinn @ July 08 2003,9:46 pm)
Thanks sponge. I think we had this problem once before and it may be caused by editing the cache at this end.
We will look into it and see whats going on here  ???

Never an issue like this with the XML.
Were it caused by an error when editing the file at this end it would at most cause it to appear it had been placed on the EDIT date instead of the PLACED date, not make it appear it had been placed the previous month. All the XML does is retrieve entries from the database without any modification and as is apparent on the cache page and checking the database, the dates are correct.

I still find no problem on this end (though I will continue looking of course).


--------------
'PC Medic'
Back to top
Profile PM AOL YIM 
Scout Offline
Cache Master




Group: Members
Posts: 721
Joined: June 2001
    Posted: July 09 2003,9:07 am QUOTE

A similar problem did show up before. I don't remember the details and I don't know the code, but I assumed it was a coding problem where the month was recorded somewhere as a value of 0-11 but somewhere else was interpreted as if the encoding was 1-12.

--------------
Scout  ( http://GPSgames.org )
Back to top
Profile PM 
PC Medic Offline
Cache Master




Group: Super Administrators
Posts: 1450
Joined: Dec. 2001
    Posted: July 09 2003,10:18 am QUOTE

Quote (Scout @ July 09 2003,9:07 am)
A similar problem did show up before. I don't remember the details and I don't know the code, but I assumed it was a coding problem where the month was recorded somewhere as a value of 0-11 but somewhere else was interpreted as if the encoding was 1-12.

I believe the issue you (and Quinn) are thinking of was in the cache script that was causing the "date placed" to change when ever a cache owner updated their cache description. This was about a year+ back. This issue Infosponge is referring to is when he is attempting to search the database using the XML script. No changes are made to the date and it simply passes the date as entered in the database in this case. Thing to remember is that all date data is in yyyy-mm-dd format. Though even that should not cause the results he is seeing.

--------------
'PC Medic'
Back to top
Profile PM AOL YIM 
infosponge Offline
Navicacher




Group: Members
Posts: 12
Joined: June 2003
    Posted: July 09 2003,11:51 am QUOTE

It's definitely acting as if there's a zero-based date vs. a one-based date somewhere.

If I dump the dates from the XML file, the most recent ones are consistently off by a month for the ones I checked:

N00714 listed=2003-06-08   (should be 07-08)
N0070E listed=2003-06-05   (should be 07-05)
N00692 listed=2003-05-08   (should be 06-08)
N00691 listed=2003-05-08   (should be 06-08)
N00690 listed=2003-05-08   (should be 06-08)
N0068F listed=2003-05-08   (should be 06-08)
N0068E listed=2003-05-08   (should be 06-08)
Back to top
Profile PM WEB 
PC Medic Offline
Cache Master




Group: Super Administrators
Posts: 1450
Joined: Dec. 2001
    Posted: July 09 2003,6:10 pm QUOTE

Again thanks for bringing the issue to my attention and believe me I am looking into it.

Problem is that while I am able to reproduce what you are seeing, the script makes no changes to the data and the problem only seems to affect some caches and not others. So while I see there is an problem, it is an unusual one and unfortunately not an obvious fix yet.


--------------
'PC Medic'
Back to top
Profile PM AOL YIM 
infosponge Offline
Navicacher




Group: Members
Posts: 12
Joined: June 2003
    Posted: July 09 2003,6:44 pm QUOTE

Quote (PC Medic @ July 09 2003,6:10 pm)
Again thanks for bringing the issue to my attention and believe me I am looking into it.

Wasn't being pushy...just wanted to give you some more data points.   :) Didn't know if you could replicate it or not.

Let me know if there's anything I can do to help.
Back to top
Profile PM WEB 
PC Medic Offline
Cache Master




Group: Super Administrators
Posts: 1450
Joined: Dec. 2001
Posted: July 10 2003,7:21 am QUOTE

OK, looking at the script it made no sense why this was happening, and while I saw a possible quick fix I held off on implementing it while in search of the rot cause. So, it took me a while, but the problem has been corrected.    :thumbs-up

Thanks to Infosponge for bringing this to my attention and Infosponge and George for there assistance in verifying the fix.


--------------
'PC Medic'
Back to top
Profile PM AOL YIM 
beej71 Offline
Navicacher




Group: Members
Posts: 2
Joined: Aug. 2003
    Posted: Aug. 25 2003,2:56 pm QUOTE

You know what else would rock in there: cache type and cache size... :)  [Lame--it's already in there, I see it now!]

Edited by beej71 on Aug. 28 2003,3:24 am
Back to top
Profile PM WEB 
The Diver Offline
Navicacher




Group: Members
Posts: 1
Joined: Jan. 2003
    Posted: Oct. 08 2003,5:25 pm QUOTE

Could the waypoint name be included?

Thanks,

Alan
Back to top
Profile PM 
skydiver Offline
Navicacher




Group: Members
Posts: 1
Joined: Oct. 2003
Posted: Oct. 15 2003,7:25 pm QUOTE

Just discovered this feature after reading about it at opencaching.com and have a question for the admins....
If I use the NewOrModifiedSince=2003-10-13&logs=1 argument, do the logs listed include all those POSTED since the 2003-10-13, or just those whose log/found date entered by the user is within that time period.  In other words, if the date in my argument is for the last two days, will a "found it" log entered yesterday for a find last week be included or not?

Thanks!
Back to top
Profile PM WEB 
Volvo Man Offline
Navicache Dude




Group: Members
Posts: 65
Joined: Oct. 2003
    Posted: Nov. 19 2003,10:19 pm QUOTE

This is a great feature, I've used it a few times, most notably to generate a world map of Navicaches in MS S&T, just needs a conversion from XML to XLS. Next I'm working on importing the whole lot, logs and all to Access, so I can have a comprehensive mobile database of caches, and then I can just set up a weekly macro to update the latest caches, and reduce the load on the site.

BTW, not sure if this has been answered or not, (getting a little tired) does the database have a date submitted field seperate to date placed?

Now the cheeky question, can anyone point me in the direction of a script to grab all the waypoints off of GC.com automatically?, might save me the effort of doing my own. PM me with answers to that if you'd rather.

Cheers, Volvo Man


--------------
Life is a succession of choices, choose wisely when you can as most of the choices are not made by you.
Back to top
Profile PM AOL 
PC Medic Offline
Cache Master




Group: Super Administrators
Posts: 1450
Joined: Dec. 2001
    Posted: Nov. 20 2003,6:46 am QUOTE

GC.com makes it a little more difficult as it is a feature reserved for their "Premium" subscibers (and even then a max 500 caches can be pulled at a time). While not a charge type service, due to some abuses we also will be placing some controls and limits on the retrieval in the near future. Seems that some are able to read data but not the TOU.

--------------
'PC Medic'
Back to top
Profile PM AOL YIM 
Scout Offline
Cache Master




Group: Members
Posts: 721
Joined: June 2001
    Posted: Nov. 20 2003,9:08 am QUOTE

Quote (PC Medic @ Nov. 20 2003,5:46 am)
While not a charge type service, due to some abuses we also will be placing some controls and limits on the retrieval in the near future. Seems that some are able to read data but not the TOU.

Can you elaborate? I'm pulling all the data nightly in order to publish my Navicache Stats. This is not in violation of the TOU as I understand them. In any case, Navicache is free to take ownership of my script and publish stats themselves if they so wish.

I could reduce the load on the server with two improvements to the XML download. First, offer a way to suppress the inclusion of all the description text for both caches and logs. It is easily the biggest part of the file and  I don't need it or even look at it. Second, provide access to an "date updated" field, so I only need to pull updated caches, not the whole database each night.


--------------
Scout  ( http://GPSgames.org )
Back to top
Profile PM 
PC Medic Offline
Cache Master




Group: Super Administrators
Posts: 1450
Joined: Dec. 2001
    Posted: Nov. 20 2003,10:16 am QUOTE

Quote (Scout @ Nov. 20 2003,9:08 am)
Quote (PC Medic @ Nov. 20 2003,5:46 am)

Can you elaborate? I'm pulling all the data nightly in order to publish my Navicache Stats. This is not in violation of the TOU as I understand them. In any case, Navicache is free to take ownership of my script and publish stats themselves if they so wish.

In an open forum I will not elaborate any more than to say your use is not (as you have noted) in violation. I will also add that while it is monitored, current server load is not the major player in this decision.

--------------
'PC Medic'
Back to top
Profile PM AOL YIM 
Scout Offline
Cache Master




Group: Members
Posts: 721
Joined: June 2001
    Posted: Nov. 20 2003,1:11 pm QUOTE

I found this in the terms of use:

Quote

"NaviCache ("NaviCache.com"), maintains this site (the "Site") for your personal entertainment, information, education, and communication. You are free to browse the Site, and you may download material displayed on the Site for non-commercial, personal use only, provided you retain all copyright and other proprietary notices contained in the materials. You may not, distribute, modify, transmit, reuse, report, or use the contents of the Site, including any text, images, audio, or video for public or commercial purposes without prior permission."


I am downloading material (using the XML download), modifying it (by creating stats), then distributing the results (on my stats Web page) for public use (i.e., non-personal). This would be a violation of the terms of use if I didn't have permission from Navicache. From PC Medic's previous comment and other previous communications that I haven't saved, I believe I have that prior permission.

Others beware. If you download Navicache data, don't republish it in any form without permission.


Edited by Scout on Nov. 20 2003,1:13 pm

--------------
Scout  ( http://GPSgames.org )
Back to top
Profile PM 
PC Medic Offline
Cache Master




Group: Super Administrators
Posts: 1450
Joined: Dec. 2001
    Posted: Nov. 20 2003,6:40 pm QUOTE

Quote (Scout @ Nov. 20 2003,1:11 pm)
From PC Medic's previous comment and other previous communications that I haven't saved, I believe I have that prior permission.

This is correct. You have received permission and have used the data as outlined. In layman terms ... Your Cool Man  :2cool

By the way, the data file that is created contains additional info and Terms Of Use. Simply put, read the terms and if you have questions, ask. We are easy to work with but some appear to be straying from the intended or agreed upon use and this can not happen.


--------------
'PC Medic'
Back to top
Profile PM AOL YIM 
GeoRockers Offline
Caching Maniac




Group: Members
Posts: 167
Joined: Feb. 2002
Posted: Feb. 12 2004,7:32 am QUOTE

Greetings everybody!

Quote
Could the waypoint name be included?


Waypoint names are now included. :)

Quote
If I use the NewOrModifiedSince=2003-10-13&logs=1 argument, do the logs listed include all those POSTED since the 2003-10-13, or just those whose log/found date entered by the user is within that time period.


This was a bug that was fixed in the newest version .

You will now get all caches _and_ logs added or modified since the target date.

All logs prior to Feb 05, 2004 have been flaged as modified on Feb 05, 2004 00:00:00.

Quote
does the database have a date submitted field seperate to date placed?


Yes for the cache table, and it was recently added to the log table.

There are three new arguments available for use:
 
   LogsOnly=1 (default is 0) - returns log entries.
   LogStatsOnly=1 (default is 0) - returns log stats only.
   gzip=1 (default is 0) - returns the data in gzipped format.

Please note that the DTD has changed, so you may want to check out the comments/history along with the DTD.  :)


George


--------------
A 1/1 via the trail around the hill?
Nah... that's too easy!  Let's go up and over!
Why follow a trail when we can blaze our own? :p

We *make* mountains out of mole hills!
Back to top
Profile PM 
Scout Offline
Cache Master




Group: Members
Posts: 721
Joined: June 2001
    Posted: Feb. 12 2004,10:39 am QUOTE

Welcome back, GeoRockers, and thanks for the enhancements. I'll be playing with them over the next few days for the stats pages I maintain. It'll be a slow process because I only run the script four times a week. Last night's run failed, so I'll try to debug it. During this process, the stats page will be screwed up.

--------------
Scout  ( http://GPSgames.org )
Back to top
Profile PM 
Scout Offline
Cache Master




Group: Members
Posts: 721
Joined: June 2001
    Posted: Feb. 12 2004,12:35 pm QUOTE

OK, I discovered my problem. Before, I used "logs=1" to get cache details and log entries. This gave me what I wanted, but a lot more too, namely all the user text of logs.

I asked for an option to suppress getting the text of log entries, because that's the biggest part of the data and I ignore that anyway for stats generation. You provided an argument "LogStatsOnly=1". Although this suppresses the log descriptions, as expected and wanted, it also suppresses the cache information. What I would like is an option "LogStats=1" that works exactly like "logs=1" but suppresses the log_entry field of log entries.


--------------
Scout  ( http://GPSgames.org )
Back to top
Profile PM 
56 replies since Aug. 26 2002,6:48 am < Next Oldest | Next Newest >

[ Track this topic :: Email this topic :: Print this topic ]

Pages: (2) < [1] 2 >
reply to topic new topic new poll

Quick Reply: XML cache data
iB Code Buttons
You are Posting as:
Guest
Do you wish to enable your signature for this post?
Do you wish to enable emoticons for this post?
Track this topic
View All Emoticons
View iB Code