Wallet Manager Project Code
Posted: 2012-10-22 Filed under: industry, market | Tags: api, industry, trade 65 CommentsIf you haven’t been following this blog, I’ve periodically posted about a “Wallet Manager” project that myself and my trading partners, Raath and James, have been working on for the past two years to help us manage our Eve activities using the Eve API.
The project started small, with the first page being driven only by the wallet API feed. As we expanded our industrial scope in the game so too did the Wallet Manager’s requirements. Soon I needed to keep track of blueprints, assets, capital production, PI, and tech 2 invention; each new requirement expanded the original scope.
What grew was a project that contained many methods by coders of different proficiencies. James took my undergraduate C/C++ knowledge and showed me how to work with more objects, models, JavaScript, and even Ajax. Over the course of two years we spent many hours developing methods to handle assets (when items get stacked their unique ID changes), minerals pricing, API caching, and ultimately usability.
The code backend is not something we are extremely proud of, but the presentation and workflow has suited our operations well. If you have the fortitude to tolerate our design, then feel free to give our project a test run.
Disclaimer *IMPORTANT*
- Code is provided as is.
- If you want to try to setup your own instance, I would advice that you have an understanding of PHP, MySQL, and general code architecture. Getting this to run is not a beginner project.
- I cannot provide any commitment for bugs corrections, adding features, or providing support.
- There are many, many non-sanitized input points that are major security issues as noted in this post. I would not use this as a public-facing site.
- There are unfinished pages, hacked together solutions, code in the view, display code in the controllers, etc. — don’t expect clean, production quality code.
- Be prepared to edit the database table to enter your users and API keys. The ‘admin’ section is only partially completed.
Prerequisites
- MySQL, PHP5+
- Yii Framework
- phpMyAdmin to efficiently work with the database
Installation
- Download the ZIP here. [April 2014 Update] I’ve moved the project to GitHub here.
- Extract the code to your var/www folder
- Edit protected/config/main.php.changeme
- Line 29 contains IP limitations for working with Gii. Enter your WAN IP address here if you want to work with this module.
- Line 57/58 contain your database connection information.
- Rename to main.php
- Edit protected/config/console.php.changeme
- Line 22 contains the database name. My project was named after my holding corporation PROHD so the database name is ‘prohd’.
- Line 24/25 contain the user/password to login to the application.
- Rename to console.php.
- Modify Yii path in index.php
- Change the path to your Yii framework location $yii=’/usr/local/lib/yii/yii.php’;
- Replace /usr/local/lib/yii/caching/CDbCache.php with the one in the zip.
- Create database structure with prohd.sql
- Create a login in the ‘accounts’ table. The default userLevel is ‘1’.
- Enable the API feed by adding a 1,1 row to the apiStatus table.
- Define a group in trackingGroups table.
- Add your character information in the ‘characters’ table.
- Import the Eve static data tables from https://www.fuzzwork.co.uk/dump/
- Import the following table into typeBuildReqs for Tech 2 items. You can generate this table using my query or import from the previous table export.
Recommendations
If you are intimated by the scope of the setup procedure, I would recommend that you use the DRK Industry Tracker for your construction projects as my partner Raath is actively developing this project.
Update
https://k162space.com/2013/08/13/wallet-manager-code-update/
k162space 
I literally said “squee” under my breath at work when I saw this. Thanks guys, will let you know how I go getting it running.
Same for me, will try to set it up asap 🙂
+1
I have been coveting this thing for a year. Now all I need to do is L2SQL and L2Program….
Looks really cool thanks o/
I got the database up and running, but i get this error once i’m connected : ( ! ) Parse error: syntax error, unexpected end of file in C:\wamp\www\prohd\protected\views\mainPage\index.php on line 404
Any clues ?
There is no line 404. I would replace that file with the one from the zip just in case the file got altered somehow.
haha ^^ I just added a blank line when trying to find the problem :p, the unexpected eof just point to the last line of the file.
I just noticed that on the login page, there is some un-interpreted php, see : http://pastebin.com/jA0UCkDJ
The issue is probably coming from here
Check your WAMP configuration. Those lines should definitely be interpreted. Make sure you have PHP5+ and have everything configured to support short PHP tags (ie “<?" instead of <?php"). Doesn't look like you have short tag support on.
This has not been tested on a WAMP setup. Currently we have it running on Ubuntu 12.04.
Could this be due to Unix vs Windows EOF characters?
Hi, and thank you for sharing this project. I was trying to get it installed, and finally passed the login screen, but i get an fatal error “Fatal error: Call to a member function getAttribute() on a non-object in /home/user/public_html/mycorp/protected/components/EVEXMLData.php on line 93”
I followed the setup instructions except the changes regarding IP limitations for working with Gii. As i setup the project on an webspace (e.g. http://www.example.com/mycorp) i was not sure what ip should be there. I changed all occurencies of prohd to mycorp also the foldernames. I hope this was correct. And as i setup on webspace there is no var/www folder, as occurs in the two cache.db files. So any idea what ist going wrong with my installation ?
I enabled php short tags, got past login too and i am facing the same fatal error
I’m running it with wamp and the latest php/apache
I’m also getting lots of undefined variables errors on the different pages such as
Undefined variable: profitData C:\wamp\www\prohd\protected\controllers\MainPageController.php(333)
on the mainPage
I passed the error by commenting the two lines which fetch the cache, so there will be empty cache to get trough, but i think i have a major issue regarding the fetch of my api data.
I can access the “menu bars” above to select orders or transactions but any selection resulöts in PHP Warning, due to empty data i think. In the admin section i can lookup my user and api data, but i’m not sure if everything is setup right. Although on the right upper corner appear API status messages, updating API, and API Update complete, but still no data in the database, so i’m still running blind…. 😦
easy about what two lines you are talking ?
I have the same issue (error 500 and the same message). And Blake, that guide above requires much more work to get it working 😉
For example, to feed the tables with static data from origina; db_dump, you need to:
INSERT INTO dgmTypeAttributes SELECT * FROM eve_inf12.dgmTypeAttributes;
INSERT INTO eveGraphics SELECT * FROM eve_inf12.eveGraphics;
INSERT INTO eveIcons SELECT * FROM eve_inf12.eveIcons;
INSERT INTO invBlueprintTypes SELECT * FROM eve_inf12.invBlueprintTypes;
INSERT INTO invCategories SELECT * FROM eve_inf12.invCategories;
INSERT INTO invGroups SELECT * FROM eve_inf12.invGroups;
INSERT INTO invMarketGroups SELECT * FROM eve_inf12.invMarketGroups;
INSERT INTO invMetaGroups SELECT * FROM eve_inf12.invMetaGroups;
INSERT INTO invMetaTypes SELECT * FROM eve_inf12.invMetaTypes;
INSERT INTO invNames SELECT * FROM eve_inf12.invNames;
INSERT INTO invTypeMaterials SELECT * FROM eve_inf12.invTypeMaterials;
INSERT INTO invTypes SELECT * FROM eve_inf12.invTypes;
INSERT INTO mapDenormalize SELECT * FROM eve_inf12.mapDenormalize ;
INSERT INTO mapRegions SELECT * FROM eve_inf12.mapRegions ;
INSERT INTO mapSolarSystems SELECT * FROM eve_inf12.mapSolarSystems ;
INSERT INTO ramTypeRequirements SELECT * FROM eve_inf12.ramTypeRequirements ;
INSERT INTO staStations SELECT * FROM eve_inf12.staStations ;
For the warnings: Run with E_ERROR only. The code is way ugly. Version 2 will be using the Play 2 framework (Java/Scala, asynchronous, non blocking, very distributed, yadda, yadda). I’ve had enough experience with PHP and Yii on this project (and others) to learn that particular setup is not very scalable, and also very hard to maintain at the sheer amount of code we have in the project. The basics of the project were very sound, but doing things that were even a little outside the Yii framework required lots of hack type work, so we devolved into admittedly very lazy coding practices as time went on. The stuff that works, though, works well (hack code aside), and is very ergonomically friendly.
As for setting up getting the API to pull and the whole thing to work correctly, a couple things.
* Password hash is in the format MD5(salt+password).
* Make sure you have at least 1 character in the ‘characters’ table.
* You will need at least 1 group defined in ‘trackingGroups’. Make sure the trackingGroupID matches the one for your account in ‘accounts’. Also, make sure the accountID in ‘trackingGroups’ matches your account.
* You will need to add at least one character to this tracking group. Add the character you defined in the characters table to ‘trackingGroupMembers’. A major caveat: characterID in trackingGroupMembers actually maps to walletID in characters.
Hope that helps.
Thanks, i got it working o/
Now i just have a billion “division by zero” warnings because I don’t have buy prices on most of my items (i do mainly industry) ;p
Hi,
thanks for your hints. I’m not yet there^^
Let me explain what i set and where i hace doubts.
I added a row to accounts, the accountID was set automatically to 4, my username is a random name, not related to EvE, just my username for THIS tool. userLvel 1 and trackingGroupID 1, fullName, my EvE Accountname, i think it is not necessary, but was a try.
I added a row to ‘characters’. walletID was set to 17 automatically. I set my EvE Character Name into the characterName field, keyCode and vCode to acces the API, the characterID i set to the characterID from within the EvEAPI setting, is this right ?
accountID set to 4.
trackingGroups, inserted a new Grp (‘MainGrp’) with trackingGroupID 1 and accountID 4
trackingGroupMembers, added a row with trackingGroupID 1, memberID was set to 14, and set characterID to 17 the walletID number which was set in ‘characters’ table.
apiStatus apiID 1 and status 1
Accessing “personal/sellorders” from the menu gives an empty table, but no errors, and my eve picture/icon is displayed, so assume the connection was made successfully, at least for the main details.
Although, selecting overview gives an error: “array_reverse() [function.array-reverse]: The argument should be an array ”
Selecting stock, gives “Division by zero ” PHP Warning
Selecting Assets gives “Invalid argument supplied for foreach()” in APIAssetList.php(33)
Selecting CHaracters gives an fatal error: Fatal error: Call to a member function getAttribute() on a non-object in /home/”user”/public_html/mycor/protected/components/EVEXMLData.php on line 25
Reports also gives an error.
thanks for your support
I’m having trouble following your setup exactly, but it seems to be ok if I’m reading it right.
Definitely disable all PHP warnings. As mentioned above, the PHP error level should be E_ERROR only. A lot of those will go away.
The biggest problem you all are going to have is that the code evolved as the database evolved, and thus large swaths haven’t really been tested at all on an empty database. At the very least, going to the overview once you are able to pull from the API should bootstrap everything.
yhup, i have the same problem with commercial coding i do,
We start with a nice defined brief, we build our models, we agree on some stuff that’s going to happen and for how much, then get to work,
Halfway through client goes, “you know how we asked for x, can you make it do Y instead?”
and i like your work “how about adding all the extra imaginery stuff we alluded too, but agreed wouldn’t be part of the current scope of the project, but can we have that any ways, but oh yeah deadline stays the same, and no extra money, Because its only an extra button right?”
and this is typically when the hodgepodge adaptability comes in.
I Havant looked at the code as yet, im more interested in the algorithms on what your formulas are for forecasting, profit calculation and things like cost averaging.
I would say our stuff is really really simple. Firstly, we do zero forecasting. On the reports page we do a couple moving averages of current data, but that’s as complex as it gets.
As far as getting profit and costing, that’s also pretty simple. We get costs in a FIFO basis (first in, first out). As purchases come in through the API, they go to the ‘inventory’ table. As things are sold, the quantities are deducted from the items in that table and the profit calculated on the original purchase cost. This allows for a pretty accurate profit calculation.
The caveat is that method is pretty prone to drift. You might buy 1000 heavy missiles you don’t plan on selling, but they’ll end up in the table. However, we two ways of combating that. One is that the tool works best when the characters being tracked do nothing but trade or produce. Because the Eve API doesn’t allow you to track specific instances of items well enough, that’s really the first line of defense. You can always spin an alt to buy your toys and keep them off the tool’s books.
The second way to combat drift is to mark items as ‘personal’. If anybody manages to get the thing to work, you’ll notice that on the transactions page, there’s a little folder/bookmark looking icon for every item. Clicking it will mark an item personal, which takes it completely out of the inventory/profit calculation loop.
Because the Eve API does basically nothing to help with tracking individual items, that was really the best compromise we could come up with.
Yep, that last 1,1,1 record in trackingGroupMembers
..helped me run the program. I’ll write an extended manual to cover the entire installation process 😉
Hello Lukas,
by no means i am a computer wizz 🙂
Did you manage to write the extended manual already? This kind of idiot guide woule be quite helpful for a lot of guys here, i guess.
many thanks in advance,
Deluca
Unfortunately not, I was not able to get it to work in the end. A few modules work, but most do not. The Wallet Manager is great, but is highly non-portable (like most organic code – some of my web apps have the same problem)
Some Extra notes for people trying this on ubuntu. You also need to install php5-curl, php5-mysql, php5-sqlite, php5-xdebug. For other systems I’m sure you can figure out the appropriate packages.
I seem to have it working okay, been able to update a few files to add in extra info I want to track, but the API Transactions updater seems to be giving errors and I can’t find why.
http://cl.ly/text/300h0O191Z1O
It says the SQL syntax is wrong, but it’s updating a few items at a time so I don’t think it is. I maybe get four or five entries of data rather than all the new data. The latest transaction it’s pulled is 10-26 18:50, but the last EVE transaction was10-28 17:14, so all the data is two days out of date. Any ideas on where I can look for fixing?
Make sure you’ve set up the characters/groups/groupMembers correctly. If the array in the SQL query isn’t populated, your setup isn’t correct.
I found it, my setup was correct, there was a value missing in APITransactions.php on line 231. It should read “$this->subtractStock($orderRow,$quantity,$walletID);”
Hi,
With help from @clanoriana, I succeed in setting this up on my wamp plateform (by the way, for wamp user : don’t forget to activate SSL for the EVE API calls, by activating the ssl apache module and the ssl php extension).
But I’m encountering some problems :
– It doesn’t find my bpc (even with assets listing them).
– I can’t start a build project for subcapitals (page list all capitals, but no subcapitals).
And I have a question :
how can I create a new build project (for modules/ subsystems) ?
Thanks a lot for the code,
Jivane
There is no facility for BPCs to come automatically out of the assets table, because EVE does not supply any BPC details (other than one exists). You’ll have to add them to the ‘blueprints’ table manually.
I don’t think we ever finished the subcaps build stuff.
True about the subcapital page – I never finished it.
I’ve built an add on to the blueprints page so they can be added via the web interface rather than via the database manually. http://cl.ly/1x2J1f313V33 Replace the protected/views/blueprints folder with the one in the cl.ly link.
The edit or delete fields don’t work yet. The NPC value is just the base price, if your value field is lower than the NPC value, then it will use the NPC value, if it’s higher, it will use your value.
(also, I’ve been playing with the theming, so things might look skewy, quick browse of the CSS code should fix it)
Also, forgot to add, I’ve not been using the Yii controller, I’ve been connecting to the DB manually, so you’ll need to create a DB connection script and save it in the root of the prohd install. 🙂
Finally i restarted on a fresh Xampp Installation and got it quite good now.
I can login and i can see some data, but the data seems incomplete.
My total balance is actual ISK, but transactions are 2-3 days old, and many data is missing.
I can see, that the API seems to work, at least partially.
But in the overview all the api fetches return “no response”, although i get some data.
Any idea where i made have messed up the db entries ?
Make sure you have a PHP with cURL enabled and make sure you followed step 9 of the install.
cURL support enabled
cURL Information 7.24.0
And the row is set correctly in apiStatus.
Today i have new data from 27.10. (yesterday only fro 26.10.) Still no actual transaction/wallet/assets data, but something was updated, but with a 2-3 day delay, strange.
I use the same API key i use for EvEHQ and there i have access to all up-to-date data.
Does anyone else have or had the same problem ?
If it’s only pulling a few tranactions at a time, see my comment from above, there was something I had to edit for it to pull all the transactions properly.
Good job Blake, I will set it up once I get back home from work. have you ever thought about publishing it as open source on github for example? Lot’s of EVE players are also software developers.
Since it’s PHP based, sources are already available anyway 😉
I am having an issue with logging in. For the accounts table, should the password, and salt both be just plain text? and it looks like the salt max length is 10, so just any 10 characters?
In the accounts table, set your UserName, the password is MD5[salt+] eg if your salt is honeynut and your password is loops, then put honeynutloops into a MD5 generator and save the result into the password field. The salt is whatever you used in the salt.
Fatal error: Call to a member function getAttribute() on a non-object in /var/www/vhosts/u2125269.plsk.regruhosting.ru/prohd/protected/components/EVEXMLData.php on line 93
tell me, what could be wrong?
I’ve got it up and running as best as I can tell but it doesnt appear to be retrieving asset info. It grabs my pilot image and my wallet balance and I can see the transactions and Corp/Alliance and skill info. Just not asset information. Any ideas?
I found loading the static data dump data fixed this issue for me, but I’m not sure exactly which tables from it you’ll need.
I install this web apps now.
But the following error is displayed. I want you to tell me what should I do.
————————
CDbException
CDbCommand failed to execute the SQL statement: CDbCommand failed to prepare the SQL statement: SQLSTATE[HY000]: General error: 1 table YiiCache already exists. The SQL statement executed was: CREATE TABLE YiiCache
(
id CHAR(128) PRIMARY KEY,
expire INTEGER,
value BLOB
)
/var/www/html/eve/industry/yii/framework/db/CDbCommand.php(354)
————————
i install this web app.
But the following error is displayed. I want you to tell me what should I do.
Parse error: syntax error, unexpected $end in /var/www/html/eve/industry/protected/views/mainPage/index.php on line 403
[…] Wallet Manager Code […]
[…] or EMMA are all good solutions, but I spent a lot of time flexing my PHP muscle to build my own wallet manager. Goonswarm has built their own import tool called Goonmetrics for people that lack […]
[…] strongly caters to our manufacturing wing, and I released my Wallet Manager that has a lot of reporting and profit […]
The download link seems broken, is the code still available somewhere?
I cant access the zip file. It looks like it has been removed. Can some one pliz email it to me on et@elite.ug
It seems the download link is broken. Is the source code still available?
I’m experiencing the same problem. Really hope it will be fixed soon, very much want to try this code out for myself.
Update here https://k162space.com/2013/08/13/wallet-manager-code-update/
[…] of software. I want to write guides for Lockefox’s Eve-Prosper tools and the once active Wallet Manager program that I have blogged about; once I learn how to create a public repository on GitHub, I […]
[…] Wallet Manager Project Code […]
You have forget to remove exploit code in one file. See git issue
Corrected
Hey man,
i’m workin on this right now. Get it run on a WAMP enviroment. Can login, see my Pictures and my Balance, but no other things. API Status says No Response on each line.
MYSQL Data:
accounts:
accountID: 1, …, userLevel: 1,trackingGroupID: 1
apiStatus:
apiID: 1, status: 1
apiID: 2, status:1
characters:
walletID: 1, …, accountID: 1, …
walletID: 2, …, accountID: 1,…
trackingGroupMengers:
memberID: 1, characterID: 1, trackingGroupID: 1
memberID: 2, characterID: 2, trackingGroupID: 1
trackingGroups:
name: default, trackingGroupID: 1, accountID: 1
in my understanding, this seems to be right. curl is also active.
Any idea, what could be wrong?
Mike
Hello! I’m trying to set your project up. At the moment things aren’t looking pretty. Anything dynamic doesn’t work, except from admin page (I can edit a user etc). I’m debugging the code, so here’s a couple of questions:
1. What’s the meaning of walletID in characters table? At first glance it looks unrelated to anything else. It’s used in the code extensively, though, and surprisingly interchangeable with characterID. For example, APITransactions::storeData($walletID) calls (inherited) EVEXMLData::getEVEData($walletID) when getEVEData expects $charactedID argument — ??? I tried to set walletID == characterID in the table, but it didn’t help much.
2. Is it going to work without static import? http://zofu.no-ip.de/ looks dead, and it’s unclear which schema is OK for the purposes of your app.
http://zofu.no-ip.de/ no longer exists, is there another static dump that can be used?
nslookup zofu.no-ip.de
Server: ***.***.***.***
Address: ***.***.***.***#53
Non-authoritative answer:
*** Can’t find zofu.no-ip.de: No answer
-Donald
yeah use https://www.fuzzwork.co.uk/dump/
http://zofu.no-ip.de/ appears to no longer exist, is there another static dump that can be used?
I got this installed this afternoon and can login and get rendered pages, however the API never seems to update and I have no data
Here is a screen shot of my overview page
http://prntscr.com/6vi9n7
accounts table;
http://prntscr.com/6via96
characters table;
http://prntscr.com/6vib63
(not sure what the characterID field is here, is it my toons pilotID or somehow tied to another table?)
apiStatus table
http://prntscr.com/6viaix
trackingGroupMembers table
http://prntscr.com/6viamt
trackingGroups table
http://prntscr.com/6viarw
I’m not where where I might have gone wrong, any help would be appreciated.
Thanks
As the original Github project no longer exists, I have created a fork and modified the code so it is working again. Updated the nstallation instructions as well. If anyone is interested, you can find it here: https://github.com/maxs94/prohd