10/17/15, 10:37 PM | #1 |
|
Introducing: ESO Market Data Relay
Hello Folks!
Alpha download available here. I have been working the last few weeks on a replication of EMDR (Eve Market Data Relay) for ESO. I am currently around alpha on the relay and looking for feedback. My focus now is verifying the data formats and working on an example site (http://www.antihax.net) on what is possible using the data. In a few days I hope to have the addon and uploader packaged for alpha testing. What Is This?! Short version; the data relay allows users to automatically upload market data as they observe it in game. Developers can connect to the relay and receive data in real time as the users change zone, /reloadui, logout, or quit game. What's the point? To allow third party development of market related apps and sites to flourish through shared interest which in turn creates useful tools and resources for players such as:
What do you need? To start, i will need testers and other developer's ideas and thoughts to build upon work already done. I'm Interested, what do i do? Contact me in game @croakroach or reply with constructive feedback. Checkout the code below
FAQ
Last edited by antihax : 02/28/16 at 08:13 AM. |
10/18/15, 08:38 AM | #2 | |
|
Even replacing the "timeRemaining", "secsSinceEvent" and similar parameters with solid timestamps would simplify detection of duplicates. |
|
10/18/15, 02:30 PM | #3 | |
|
|
|
10/19/15, 02:52 AM | #4 | |
I think your stance that the recipient is responsible for handling duplicates might be a huge turnoff for many developers, as this is one of the most difficult problems in here and if everyone has to solve this problem for themselves, you will also get a lot of duplicate solutions.
Rounding the timestamp to 10 is a bad idea, because you will a) loose precision and b) the timestamp varies by a few seconds between session. In the past I also started writing something similar to get store data out of the game and entries that had a stamp that ended on a 5 in one session had one ending on 4 or 6 in another which would then round differently. I had an idea of using the order of the entries to identify duplicates, but never implemented it. Search for the entry by timestamp and then see if the previous and next entries have the correct item ids. If there is one wrong, it would be a duplicate. There is also the problem that you will have to handle manipulated uploads in some way. When you allow anyone to upload data, anyone can manipulate your data freely in any way they want by sending in fabricated entries.
Getting data into the game is even harder and in my experience most ESO players won't use any data source outside of the game while playing. That's why I gave up my own project which did exactly what yours is doing and started making AwesomeGuildStore instead. |
||
10/19/15, 07:14 AM | #5 |
|
Good feedback!
I did see yesterday that zone changes do not count. Will have to find some trick to get this to change. I think I misworded about the duplicates. The system is intended to send duplicates, how do you know data is valid in an anonymous system? How do you calculate probability that an item is still listed? What if someone connects after an item was observed? This is what sending duplicated data is for. I agree a better solution is to have a unique Id, but we do not. I'm going to see how well rounding to 10 works. Not to worried about losing 10 seconds. What other options are available cross client? |
10/19/15, 10:01 AM | #6 | |
|
in what format is it uploaded? You can ask for quite a bit about a sale and put all that info together (with a separator so you can split it later) and get a fairly useful ID in my previous post I said I have two routines to remove dupes I parse a specific savedvariabls file and each sale is put into this format: Code:
Amazing Deals of Tamriel,Argentum,15,1437713676,150,@Medusanomi,@dOpiate,1:0:1:0 That is the data I need to make my program work. But if the user deleted their savedvariables and this time it comes back with a slightly different time stamp then I drop that from the string and check for Code:
Amazing Deals of Tamriel,Argentum,15,150,@Medusanomi,@dOpiate,1:0:1:0 the "details" are created by this code Lua Code:
so you can get a fairly reliable amount of data on a sale to make into some sort of identifier from. They even added lots of new info into the itemLink with the recent patch. -d |
|
10/19/15, 10:59 AM | #7 |
|
I am using a similar method as you describe in the lua side.
It looks like the saving of the saved vars file was intentionally locked due to using it as a signaling method for cheats. So I may change the approach. |
10/19/15, 05:56 PM | #8 | |
|
|
|
10/20/15, 06:26 AM | #9 | |
|
a brief history of ESO market addons...
just a little background... Shopkeeper is based off of Trade Sales History and MM is based off of ShopKeeper (the version Garkin and I fixed and updated - the core of the code was written by Khaibit and awesomebilly). Philgo68 used ShopKeeper as the base of MM, also some code from ShopKeeper made it's way into Price Tracker Updated when Garkin took it over. Master Merchant is a great upgrade to ShopKeeper and to make my addon seamlessly integrate with his, Philgo68 was kind enough to work out a system that if my addon is detected then he produces a saved variables file specific to GSA. If you aren't using GSA then this saved variables file does not exist. Why I am telling you all this? Because when you said you duplicated the MM format (It's very similar to the ShopKeeper format, but he collects a bit more information*) I wanted you to know how that format evolved and who was involved. I like to give credit where credit is due. If this is something you are duplicating from MM I just wanted to let you know how MM came to be. Master Merchant is a great addon, really a must have addon along with Awesome Guild Store (they also work together, which is a really nice feature). This post isn't so much as feedback but background info so the authors of these great addons get the proper credit for getting us to this point. I'm just providing feedback and suggestions based on my experiences from working on ShopKeeper and GSA and then the integration with MM that I hope will help you in some way. -d *Hindsight being 20/20, I wish I had added more info on each sale to make each sale as individual as possible, but right now GSA works 100% and I have no desire to make any big changes. But, I learned to never say never. We will just have to see how interesting the next DLC is. Last edited by dopiate : 10/20/15 at 06:41 AM. Reason: me fail English? That's unpossible! |
|
10/20/15, 07:23 PM | #10 |
|
Quick Suggestion
If you leave the executable in the AddOns folder it's less likely to get used. If you want you can use my Desktop Icon vbs. Put it in the zip as MakeDesktopShortCut.vbs.remove That way the user can rename it minus the .remove and run it so it puts a shortcut on the desktop. This is the way mine does it - feel free to use this if you want... Code:
Set oWS = WScript.CreateObject("WScript.Shell") Set objFSO = CreateObject("Scripting.FileSystemObject") homedrive = oWS.Environment("PROCESS").Item("HOMEDRIVE") homepath = oWS.Environment("PROCESS").Item("HOMEPATH") CurrentDirectory = objFSO.GetAbsolutePathName(".") shortCut = homedrive & homepath & "\desktop\Guild Sales Assistant.lnk" Set oLink = oWS.CreateShortcut(shortCut) oLink.TargetPath = CurrentDirectory & "\Guild Sales Assistant.exe" oLink.Save -d Last edited by dopiate : 10/20/15 at 07:27 PM. |
10/21/15, 10:56 AM | #11 |
|
Sounds like a good idea. Right now it runs at Windowsboot.
|
10/21/15, 07:10 PM | #12 |
|
Ummm, I'm a bit confused. You wrote an addon that runs at startup? Exactly how do you let the user decide if they want to do this or not? Why would any user want to have an ESO addon running at startup? Does it stay in memory? If so that is not a good idea, it's actually a terrible idea. I'll wait for your reply before I say more, but your program is a guest on the users computer. It should be specific to the game, only work to enhance the game for them and should not be have any "auto start" capabilities whatsoever outside of ESO. The end users of this game puts their trust in the developers because our only goal is to make their gaming experience better. If you wrote something that stays active all the time you are taking up resources (regardless of how small the amount) and you are no longer a guest but an intruder. -d |
10/21/15, 09:46 PM | #13 | |
|
The back end is starting to shape up and the example front end is coming along. Based on our discussions prior though, I may end up offering an hourly or daily database dump and/or an API for getting data from the database, while leaving the option for connecting to the relay open. Check: http://www.antihax.net use the search box on the left. |
|
11/03/15, 11:06 PM | #14 |
|
So bit of an update here, been adding functionality to the site and realized it is fully possible to insert data back into the game... so, done that. The latest version of the uploader will also pull weekly statistics from the site API and create a data file which is loaded by ESO. Also created some public helper functions so other developers can access this within lua.
Our major benefit here is that the more users we have, the more guild sales are observed and the more guild kiosks are seen. This is combined and fed back to addons so more than 5 guild stores histories make up the statistics. What other data can we take out, merge and put back that would be useful and not provide unfair advantages? Here is the Lua API available:
The site also has a few JSON API URLs available now. E.g.:
This is all still alpha and subject to change, but focusing more now on polishing, documenting, and adding more features where it makes sense. Also don't forget to check out the progress on the main example site. antihax.net |
11/04/15, 05:33 AM | #15 |
|
Why return week stamps as string? Numbers would be better IMO. Besides, you need to string.format("%04d%02d", 1970 + epocYears, epocWeeks) if you want correct strings for the first 9 weeks.
|
11/04/15, 06:40 AM | #16 |
|
Thanks for the feedback. I was thinking the same last night and will change it later today. Was rather late.
|
11/04/15, 06:56 PM | #17 |
|
This is done, it is now a number and expresses 1-9 correctly.
|
03/05/16, 05:18 PM | #18 |
|
I have uploaded a simple addon to access the data in game from the return feed: GuildStoreTools
This requires ESODataRelay to be installed and running. I am looking at the possibility of an MM extension to provide basic statistics, and possibly replace the graphing if i get time aside from the relay code. |
ESOUI » AddOns » Alpha/Beta AddOns » Introducing: ESO Market Data Relay |
«
Previous Thread
|
Next Thread
»
|
Display Modes |
Linear Mode |
Switch to Hybrid Mode |
Switch to Threaded Mode |
|
|