| Welcome Guest | Login |
Index
| Recent Threads
| Register
| Search
| Help
| 
| View Unanswered Threads
|
 |
Forums » List all forums » Forum: Tips & Tricks, Questions & Answers » Thread: Monitoring commodity prices over time? |
|
Thread Status: Normal Total posts in this thread: 10 |
[Add To My Favorites] [Watch this Thread] [Post new Thread] |
| Author |
|
|
Pauling
Joined: Dec 20, 2005 Posts: 1997 Status: Offline |
In order to run my shops more effectively, I'd like to get a better feel for how commodity prices vary over time. Years ago, rustycutlass.org offered a way to do this by monitoring tax prices, which you could then graph to see ocean-wide price cycles over the span of years. Is there a more modern, and still-functioning, site that replaces rustycutlass? Ideally a site that lets you download the raw data? As of last night, I've begun automated tracking of tax data for my own private consumption, but obviously it will be some time before I start accumulating enough information to show long term trends for the three oceans. ---------------------------------------- Pauling of the Lost Lot, Sagemerald Officer emeritus and occasional cabin person Puppetar (and drawing of |
||
|
|
zilenserz
Joined: Mar 15, 2007 Posts: 200 Status: Offline |
How're you automating it? I was thinking about doing a manual version for myself, but it'd be really time consuming... ---------------------------------------- Zilenserz on Hunter, Sailing round in circles sincle April '07.
|
|||
|
|
Pauling
Joined: Dec 20, 2005 Posts: 1997 Status: Offline |
I wrote a quick and dirty python script that parses the tax table with BeautifulSoup, and runs nightly on a web server via a cron job. Then, the results are written as a new line to a CSV file (one new line per day, one file per ocean). For manipulation, I loaded the data into Google Docs / Excel using importData. Of course, I only have 2 days of data at the moment, so it's pretty manageable. :) It's fine for personal use, but for a web app that others could use, I don't think that reading an entire ginormous CSV file for any of a thousand potential users is the right way to go about it. I lack experience with databases, though. As data continues to accumulate, I'd be open to suggestions, collaborations, or data sharing with more dev-savvy pirates. (I'm not averse to sharing the code- but I'd prefer to see a single centralized service rather than a dozen web server scripts that not everyone can use) ---------------------------------------- Pauling of the Lost Lot, Sagemerald Officer emeritus and occasional cabin person Puppetar (and drawing of ---------------------------------------- [Edit 1 times, last edit by Pauling at Apr 2, 2012 11:55:19 AM] |
|||
|
|
SirCarl67
Joined: Feb 5, 2006 Posts: 2086 Status: Offline |
I use this. ---------------------------------------- Apollo tells ye "If you're ok for me to do one [bake-off], I'll go with that." ---------------------------------------- [Edit 1 times, last edit by SirCarl67 at Apr 2, 2012 11:58:28 AM] |
||
|
|
wrs1864b
Joined: Apr 10, 2006 Posts: 4576 Status: Offline |
That is the only publicly available source I know of also. I don't find it very useful since you can't change the time scales and a lot of minerals vary over several years. I have spotty data for Cobalt going back to 2010, but that is only useful enough to show to me that the tax values aren't that useful. For example, tax values on Cobalt depended a great deal on whether I was active in the game, doing trade runs and selling stuff to many people, or I'm away from the game and no one else is performing that function. Tax values are a bad mixture of too many different prices, many of which are easily manipulated by players, either accidentally or on purpose. Tax values are nice to figure out what the sales tax will be for a product, but it can be dangerous to read too much into them for long term trends. Heck, even trying to use tax info or products can fail since not all tax values for all ingredients are published by OOO. The only way to get the tax values for all commodities is to have a BK holding an island, and then you can look at the fort buy/sell prices. (BKs set the buy price to tax*10*0.90 and the sell to tax*10*1.10.) ---------------------------------------- Algol can not assert the truth of all statements in this post and still be consistent. |
|||
|
|
Pauling
Joined: Dec 20, 2005 Posts: 1997 Status: Offline |
Thanks for the tip! I've been planning to fetch tax data daily, but the right-hand plots on weathered helm seem to feature far more fine grained data (and changes over smaller intervals). Does anyone know what the ideal sampling interval is? I can keep fetching data for now... tax rates aren't great, but they're better than nothing. (I suppose that PCTB's OCR-fed per-island data would be interesting to mine, but that's a bit too large a project for me to take on right now) ---------------------------------------- Pauling of the Lost Lot, Sagemerald Officer emeritus and occasional cabin person Puppetar (and drawing of |
||
|
|
wrs1864b
Joined: Apr 10, 2006 Posts: 4576 Status: Offline |
I don't know, but I know that the actual sales tax gets out of sync with my downloaded version in as little as 10 minutes, but I rarely am comparing stuff that closely and I could have just gotten unlucky a few times. I'd be tempted to pick one ocean, download it every 5 minutes for a few hours or a day or two and see.
Yarrg's downloader has an option to save the data locally, and I've been using that since late 2009. Yes, I find the market data to be far more useful than the tax data for trends. I would make my stuff public, but making it usable/understandable is a bit too large a projected for me to take on right now. :-< ---------------------------------------- Algol can not assert the truth of all statements in this post and still be consistent. |
|||||
|
|
Pauling
Joined: Dec 20, 2005 Posts: 1997 Status: Offline |
I've changed the sampling interval from every 24 hrs to every three hours for now. Incidentally: based on the above tip about BK tax rates, you actually CAN use BKs to get tax rates for every commodity at any time, if you have a spare island. On the emerald testing ocean, we discovered that BKs can be manually made governor of an island at any time, and then immediately voted right back out again by the monarch of the island holding flag. (But see this page before leaving PoE in the coffers, or otherwise prettifying the island: http://yppedia.puzzlepirates.com/Brigand_King#What_happens_when_a_Brigand_King_takes_an_island ) ---------------------------------------- Pauling of the Lost Lot, Sagemerald Officer emeritus and occasional cabin person Puppetar (and drawing of |
||
|
LJAmethyst
 Joined: Jul 19, 2007 Posts: 3121 Status: Offline |
Hmm, I wonder if you've obtained Three Rings' permission to scrape their websites like this - not sure if it is required, but it might be nice to ask if you're going to make regular dips. Also, it might be interesting for you to check Jade and Opal as well; I don't know if there is any translation involved here. ---------------------------------------- First Mate, Tales from the Deep. Prince, Jigsaw Tessellators. Governor emeritus of Amity Island. |
||
|
|
Pauling
Joined: Dec 20, 2005 Posts: 1997 Status: Offline |
That's an extremely good suggestion, and at the moment my only move has been to shy away from scraping too often (my goal is 1-2x a day if feasible). I think I dropped the ball a bit there. :) The information obtained is a trio of 9.8 kB publicly available pages, but I'll see about sending a petition this evening. The file format appears to be the same for Opal and Jade, and the commodity names on the tax page weren't even translated... but let's make sure that scraping is ok before adding more oceans to the mix. EDIT: Bia very helpfully pointed me towards the official policy on third-party tools, which may be found online. Looks like scraping is ok, within reason: http://yppedia.puzzlepirates.com/Official:Third_Party_Software#Reading_public_ocean_websites ---------------------------------------- Pauling of the Lost Lot, Sagemerald Officer emeritus and occasional cabin person Puppetar (and drawing of ---------------------------------------- [Edit 1 times, last edit by Pauling at Apr 2, 2012 6:53:57 PM] |
|||
|
| [Show Printable Version of Thread] [Post new Thread] |
Powered by mvnForum
mvnForum copyright © 2002-2006 by MyVietnam.net