Welcome Guest | Login |
Index
| Recent Threads
| Register
| Search
| Help
| ![]() |
![]() |
Forums » List all forums » Forum: Tips & Tricks, Questions & Answers » Thread: Intent to fork: PPAOCR (Pirate Commodity Trader with Bleach) |
Thread Status: Normal Total posts in this thread: 19 |
[Add To My Favorites] [Watch this Thread] [Post new Thread] |
Author |
|
alxwrd
Joined: Sep 19, 2017 Posts: 7 Status: Offline |
Ahoy! I want to publicly announce my intent to fork the project known as Pirate Commodity Trader with Bleach (PCTB). More specifically, the OCR client known as PPAOCR, as the source code for the web server was never released. I am creating this thread incase any of the original creators of PCTB are still active and have any concerns. Also, to allow for discussion in the community. The resulting project will be under the MIT licence , the same as the original. Why? A lot of trade tools currently require 'hooking' into the Puzzle Pirate client to access data from the commodity screen. My understanding is these tools use their own copy of Java with accessibility options turned on (screen readers, etc) to allow reading of the market data as plain text. However, with the release of the Puzzle Pirates - Dark Seas, I haven't found a way to get these tools to work with the Steam install of Dark Seas. As such, I will looking to revive PCTB. Problems with the original The current version of PPAOCR has a couple of issues which have caused it to stop working. OCR doesn't work on some clients Both the official YPP client from http://www.puzzlepirates.com/ and the new Dark Seas client have some changes to the header height in the commodity window, which means the OCR no longer works on those clients. It does still work with the Seam version of YPP. ![]() ![]() Upload site no longer exists The site at http://pctb.crabdance.com/ is no longer available. So in addition to fixing up the issues with the client: I will be looking into creating a server application that will also be released to allow anyone to host their own instance if they so desired. When? Unless:
I intend to create a fork of the project sometime in the next week. The project will be hosted on github. In the meantime, please let me know any thoughts/concerns or any ideas for a project name! ---------------------------------------- Fatman - Obsidian |
||
|
wrs1864b
Joined: Apr 10, 2006 Posts: 6337 Status: Offline |
Why not use YARRG? And, since Obsidian only has three colonized islands, two of which aren't used much, it really isn't critical that you need trading tools right now. ---------------------------------------- Algol can not assert the truth of all statements in this post and still be consistent. |
||
|
Sagacious
Joined: Mar 6, 2006 Posts: 4798 Status: Offline |
As for the different table headers, that's likely going to relate to the version of java being used to launch the game. ---------------------------------------- Cerulean & Meridian - Icemeister Emerald & Obsidian - incognito! #TeamPurple |
||
|
alxwrd
Joined: Sep 19, 2017 Posts: 7 Status: Offline |
wrs1864b:
I have been unable to get YARRG/JARRG to work with Dark Seas: JARRG runs the unmodified Puzzle Pirates game, but with a separate copy of Because Dark Seas requires Steam to log on, and JARRG uses it's own copy of a JVM, I couldn't get them to work together. If you have a way to achieve this, please let me know. wrs1864b:
Of course, I agree it isn't critical, but it would be useful. Sagacious:
Agreed. The current version of PPAOCR expects them to be a certain size though, which is why the screen scraper no longer works. Which is why it's my intent to make a copy of the project, to get it working again. ---------------------------------------- Fatman - Obsidian |
|||||||
|
wrs1864b
Joined: Apr 10, 2006 Posts: 6337 Status: Offline |
I have pretty much retired from the game, so I haven't investigated problems with YARRG much. In my experience, the old OCR based system was much less reliable (and slower) than the JARRG uploader (which worked for both PCTB and YARRG). It required the YARRG maintainer to constantly update the OCR system to recognize new fonts and character sizes and even then, it would get things wrong at times. (For example, sometimes you would get listings for commodities from "Mercenary Supply Shop", "MercenarySupply Shop", AND "Mercenary SupplyShop" without any indication of which was the most current data.) The maintainer for YARRG actually tracked me down in-game to complain about me still using the OCR program long after the JARRG uploader was published because it created extra work for him to do. If you are going to put some effort in to this, I highly recommend trying to get the JARRG uploader working in some form. OK, not only does JARRG not work, but YARRG doesn't support Obsidian and you can't get YARRG to support Obsidian because no one has updated the Yppedia for all the island maps yet. I keep telling myself that I should do that, but, well, I'm mostly retired. ---------------------------------------- Algol can not assert the truth of all statements in this post and still be consistent. |
||
|
alxwrd
Joined: Sep 19, 2017 Posts: 7 Status: Offline |
I actually spend wayyy too much time trying to get tools like JARRG and Trade Profiteer working with the new Steam client. I might spend some more time on this though because I do agree that the approach by YARRG is much more robust, but the main problems I encountered is replacing the java_vm in the Dark Seas directory means Steam sees the files you have locally doesn't match the files Steam has, so it causes it to patch the game. ---------------------------------------- Fatman - Obsidian |
|||
|
wrs1864b
Joined: Apr 10, 2006 Posts: 6337 Status: Offline |
Fixing PPAOCR is just a fairly small part of cloning PCTB, and for the most part, PCTB doesn't have the functionality of YARRG. There are two good points about this. First, the YARRG creator(s) have released the full website source code. Second, I have developed my own system that is in many ways much better than YARRG. But, as I mentioned, YARRG currently can't support Obsidian because the Yppedia doesn't have all the island maps and my system depends on a working YARRG. Being mostly retired from the game for about 2 years, I've been having a very hard time getting motivated to do the work needed to let others use the system I have developed. I'll try get something out soon so you don't have to re-invent the while for the 4th or 5th time. The best solution to the market data upload problem is to have GH get off their butt and export the dockside market data and the bid market data via yoweb. ---------------------------------------- Algol can not assert the truth of all statements in this post and still be consistent. |
||
|
Shinito
Joined: Jan 3, 2006 Posts: 588 Status: Offline |
---------------------------------------- Thorkill on |
|||
|
JerikTeloria
|
I am brand spanking new to the community but I was actually looking to do something similar to this using Tesseract and C#. If you'd like to start from scratch it shouldn't take too long and I can share with you the code I have so far. https://github.com/charlesw/tesseract
That said, I do really think this is the better solution. ---------------------------------------- [Edit 2 times, last edit by JerikTeloria at Sep 21, 2017 3:21:29 PM] |
|||
|
alxwrd
Joined: Sep 19, 2017 Posts: 7 Status: Offline |
Thanks! Help would be much appreciated. I will look at getting the project on github later today. I did look into Tesseract, but the current OCR client seems to be working fine. It may be an option moving forward? Just for reference: While the current client is written in C# but I'm actually a Python dev (I'm currently working with IronPython, so I have had some experience with .NET), so I was intending on creating the web app using Python. I don't know if this is something you'd still be interested in? That said, here are my current objectives:
---------------------------------------- Fatman - Obsidian |
|||
|
alxwrd
Joined: Sep 19, 2017 Posts: 7 Status: Offline |
Just a quick update. Both projects are now on github: PACS - Market Scraper github.com/alxwrd/pacs The market data portion of the client is working. I am debating if I should remove the bid data as I don't see how it's useful in a trading tool (you would need to sit on a bid ticket for a bit, at which point the dock prices might have changed). I have done a release if anyone wants to try it out. The upload won't work as it currently has no where to go. Instructions:
Data is stored as json in %APPDATA%\PPAOCR\Commodities.json and has the following format:
Merchant Almanac - Web Server github.com/alxwrd/merchant-almanac Not totally sold on the name. Not much has happened here as I would have liked, but this is my next focus. Going forward, I won't keep updating this thread with development as I don't want to keep bumping it. If you want to follow along, keep an eye on the github repos. ---------------------------------------- Fatman - Obsidian |
||
|
wrs1864b
Joined: Apr 10, 2006 Posts: 6337 Status: Offline |
Personally, I found the bid market information to be more important than the dockside market data. Since the JARRG uploaded didn't keep track of it, I used other techniques. While bid tickets aren't as immediate as gems or dockside trades, filling tickets and then shipping around the filled commodities are where the real PoE is at. That said, if you are an experienced trader, you can almost keep track of the bid market in your head. What hiding the bid market does it it doesn't open the eyes of inexperienced traders. The other comment I have is that I would f recommend is keeping a timestamp for when the data was uploaded. ---------------------------------------- Algol can not assert the truth of all statements in this post and still be consistent. |
|||
|
alxwrd
Joined: Sep 19, 2017 Posts: 7 Status: Offline |
Just want to drop in to post a quick note about potentially using JARRG going forward. I was looking through the Terms of Service to see if I can use in game art/assets and on the page about Third Party Software there's the following section:
Previously JARRG had it's own Java vm. But because Dark Seas runs through Steam to get Steam integration you would need to modify the Java vm in the directory, which wouldn't be allowed. I don't know if we could get an OM comment on this? P.s. Does anyone know what the stance is for using assest in 3rd party tools? ---------------------------------------- Fatman - Obsidian |
|||
|
wrs1864b
Joined: Apr 10, 2006 Posts: 6337 Status: Offline |
On this kind of thing, when in doubt, do a petition. Make sure you are very clear about the situation and that probably a dev not just an OM needs to consider things. That said, I recommend looking at the Official:Third Party Software. When this policy was last updated,YARRG (and PCTB) was explicitly discussed with a reply from the OM Endymion. So, I'm pretty sure that GH knows what the JARRG uploader does and is ok with it. The important thing is that the JARRG uploader neither modifies any file, nor does it modify the game executable either on disk nor in memory. Early on, JARRG used a different run time java executable on disk, but OOO convinced the JARRG developers that they need to use the standard java program that YPP uses and use the standard "screen reader" API to do all the work. So, does this reasoning still apply to Steam? I don't know, but I suspect GH will say it is OK. For what it is worth, I host an updated version of the JARRG uploader that kind of works with YPP classic. The source code on the YARRG web site broke when OOO changed the game. Now my patched version started to fail when a new version of java was released and I've never gotten around to fixing it. I don't think I've ever published the patches I made, I'll try to fix that. ---------------------------------------- Algol can not assert the truth of all statements in this post and still be consistent. |
|||
|
polaris395
Joined: Oct 23, 2005 Posts: 1330 Status: Offline |
SO I've been scraping with it and it has been grabbing the data well. Unfortunately, I was able to CTRL+C into an excel document (great feature), but now I can't do it again. I select all the data, copy it, and then all the rows of data clear. Any ideas on whether the export to excel can be made easier? Using the clipboard would be ideal. Thanks! Edit: Also, I am unable to get PV to load. It says island not found. Lionhaven loads however. Win 7 Ultimate, 64 bit. ---------------------------------------- -Polariss
---------------------------------------- [Edit 1 times, last edit by polaris395 at Oct 17, 2017 7:33:05 PM] |
|||||
|
Fendero
|
You might want to take a look at my scraping tool, which is open source and doesn't require OCR at all. http://forums.puzzlepirates.com/community/mvnforum/viewthread?thread=207575 |
||
|
wrs1864b
Joined: Apr 10, 2006 Posts: 6337 Status: Offline |
I will say again, that instead of re-inventing the wheel, people should look at YARRG. It is open source, doesn't require OCR at all, figures out the correct ocean and island by itself and does a whole lot more than just figuring out trades between two islands. ---------------------------------------- Algol can not assert the truth of all statements in this post and still be consistent. |
|||
|
polaris395
Joined: Oct 23, 2005 Posts: 1330 Status: Offline |
Ive seen it. Havent bothered to get visual studio, and lack of an executable makes laziness an issue. Since it outputs a CSV Ill look into it.
Again, seen it. Does it output a CSV? I dont really care about sharing data with others. I just need to bring all market data into excel so I can do my own thing. Looking at the website clearly Im using it wrong because I cant get any trade routes on Obsidian to actually be made. But thats an issue for that thread anyway. ---------------------------------------- -Polariss
|
|||||||||||
|
wrs1864b
Joined: Apr 10, 2006 Posts: 6337 Status: Offline |
The JARRG uploader has an option to write a "test" file of the island you uploaded, which I seem to remember being in tsv format, not csv. Excel will import either. The author is a strong supporter of open source programs and data, so it was designed to share the data too. As far as why you can't get any trades on Obsidian via YARRG, well, no one has uploaded any island data yet. If you want to see what YARRG can do, look at other oceans. ---------------------------------------- Algol can not assert the truth of all statements in this post and still be consistent. |
|||||
|
[Show Printable Version of Thread] [Post new Thread] |
Powered by mvnForum
mvnForum copyright © 2002-2006 by MyVietnam.net