The audience of this post is a person that has a database table that is populated from the /char/WalletTransactions.xml.aspx API and wishes to use queries to produce meaningful, intelligent results that can be used to help focus trading operations.
Even with limited database query knowledge, we can derive meaningful Business Intelligence out of our dataset. Basic MySQL/SQL knowledge is recommended as values or parts of an example query may need to be modified to suit your environment or table structure.
I won’t cover the design on how to get API data into your database table here as I want to save that topic for its own post. If you need an answer to populating your database from the API, search the Eve Forums as there are a many projects that can help.
Wallets from multiple characters are stored into a database table.
Business Intelligence from Numbers
Knowing that a Drake sold for 1.1 M profit one time doesn’t tell you much in the larger scheme of your business as we need to extract more intelligence out of our numbers. Getting further metrics out of the wallet data can help focus your operation and deliver better results.
Consider these starter questions: How fast have Drakes been selling (item velocity)? Are the the monthly profit from Drakes worth your time or should you be working with another item? Is working with Tech1 Battlecruisers proving profitable?
These are examples of the types of questions that you can answer with a few database queries and some spreadsheet analysis. The more robust developer will build these queries into a reporting application, but for the purpose of this post, I will stick to running a query and working with the output.
I have added a few unique columns to my table to help me filter my data. Here is my ‘wallet’ table structure for reference.
– Table structure for table `wallet`
CREATE TABLE IF NOT EXISTS `wallet` (
`transactionDateTime` datetime NOT NULL,
`transactionID` bigint(20) unsigned NOT NULL,
`quantity` bigint(20) unsigned NOT NULL,
`typeName` varchar(255) CHARACTER SET latin1 NOT NULL,
`typeID` int(11) NOT NULL,
`price` double unsigned NOT NULL,
`clientID` bigint(20) unsigned NOT NULL,
`clientName` varchar(255) CHARACTER SET latin1 NOT NULL,
`characterID` int(11) NOT NULL,
`stationID` bigint(20) unsigned NOT NULL,
`stationName` varchar(255) CHARACTER SET latin1 NOT NULL,
`transactionType` varchar(4) CHARACTER SET latin1 NOT NULL,
`personal` tinyint(1) NOT NULL DEFAULT ’0′,
`profit` double NOT NULL DEFAULT ’0′,
PRIMARY KEY (`transactionID`),
KEY `characterID` (`characterID`),
KEY `transactionDateTime` (`transactionDateTime`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
characterID, personal, and profit are unique columns that I use to help track items for reporting purposes.
- characterID – I have multiple character wallets writting to this table so I have a ID for each one stored in a characters table.
- personal – this is a 0/1 flag that I use to note if an item in the wallet is for personal use. Examples are fuel for jumping capitals or ammo for PVP adventures.
- profit – calculated profit of the item when sold minus fees.
– Total Profit per Item
SELECT typeName, typeID, SUM(profit)
AND personal = 0
GROUP BY typeID
ORDER BY SUM(profit) DESC
– Price and Profit sums per Month in 2012
SELECT MONTH(transactionDateTime) AS calMonth, SUM(price) AS price, SUM(profit) AS profit
WHERE transactionDateTime LIKE ’2012-%’
AND personal = 0
GROUP BY MONTH(transactionDateTime)
– Total Profit and Sales per day
SELECT DATE(transactionDateTime) as date1, SUM(profit) as totalProfit, SUM(price) as totalPrice
AND personal = 0
GROUP BY date1
ORDER BY date1
– Average Margin and Total Quantity per Meta Group over the past 365 days
SELECT invMetaTypes.metaGroupID, invMetaGroups.metaGroupName, AVG((profit / (price * quantity))) * 100 as averageMargin, SUM(quantity) AS quantity
JOIN invMetaTypes ON (wallet.typeID = invMetaTypes.typeID)
JOIN invMetaGroups ON (invMetaTypes.metaGroupID = invMetaGroups.metaGroupID)
WHERE transactionDateTime > DATE_SUB(CURDATE(),INTERVAL 360 DAY)
AND transactionType = “sell”
GROUP BY invMetaTypes.metaGroupID
ORDER BY metaGroupID
SELECT wallet.typeID, wallet.typeName, SUM(profit) as totalProfit, SUM(quantity), SUM(price), b.marketGroupID, a.marketGroupName, b.marketGroupName as parentName1, c.marketGroupName as parentName2, d.marketGroupName as parentName3
JOIN (invTypes, invMarketGroups as a, invMarketGroups as b, invMarketGroups as c, invMarketGroups as d)
ON (wallet.typeID = invTypes.typeID
AND invTypes.marketGroupID = a.marketGroupID
AND a.parentGroupID = b.marketGroupID
AND b.parentGroupID = c.marketGroupID
AND c.parentGroupID = d.marketGroupID)
AND personal = 0
AND d.marketGroupID = 11
GROUP BY wallet.typeID
ORDER BY totalProfit DESC
Flex Your Database Muscle
These examples should serve as a branching off point as you will want to tailor them to your operation. You may want to replace SUM with AVG or COUNT, modify the date range to take a look at a specific trading period, or hopefully roll these types of queries into a reporting engine to really drive your business intelligence.
If I had to pick a word to describe the past quarter, I would say “uninspired”.
The Retribution expansion brought little to no change that affected my industrial operations or new styles of play. Hopefully there will be industrial changes in the newly announced Odyssey expansion to bring new life to this area of Eve.
I’ve been very hands off with the Capital construction division, as Raath has taken complete control of the production and sale cycles. Personal commitments have taken over my allotted Eve time. Q1 of the year means a return to a workout cycle that prepares me for summer Triathlon and Open Water Swimming competitions. These sessions increase in intensity and duration leaving me in a wobbly, static state by the time I get home and power up the computer.
Slow and Steady
Daily trading has all but ceased as I’m relying on the Capital construction division to drive revenue. Right now our operation is performing at around +3.3 B/month, which is below my record of 8 B/month. Not bad for passive income, but not ideal.
In late February a stealth change to NPC prices hit the live server. I have not added the 10-11% increase in value to my net worth yet. Since I own around 85 B in blueprints, this will be a nice increase in net worth.
Interesting to see moon minerals on one the Odyssey announcement slides at PAX East 2013. I hope that a serious look at industry balancing is going to take place now that the general population seems pacified by the content in the Crucible, Inferno, and Retribution expansions.
The icons shown are for R64 moons, which have concentrations skewed towards galactic quadrants as shown by data analysis. I can only imagine what they have planned as they have stating their intention is to remove passive static resource fountains such as the current moon mining system.
My side interest in information aesthetics is present in my work, both professional and personal. I’ve received many comments on how clean and clear my analysis charts, reports, and spreadsheet work can be so I though I would share some of my Eve related spreadsheet work to inspire people.
Here are screenshots from depreciated spreadsheets that I used for industrial production, speculation, and fuel estimations before switching to a custom web-based solution that my partner, Raath, created.
My design goal has always been to use small type-face Arial text that is boxed and columned with important information highlighted in primary colors or shaded in gradients to denote best to worst status. Some of these sheets are old with broken formulas or outdated build requirements. Pay more attention to the organization rather than to the numerical content.
In Winter of 2012, I approached my partner Raath to see if expanding our operation into Heavy Industry was a viable means to take excess liquid capital and start to produce return for our efforts. After scouting markets, running numbers, and placing production characters in proper locations the operation formed.
We’ve reached a state of industrial nirvana as our operational inefficiencies have been reduced and logistic kinks have been worked out over the past few months. Given that we’ve nailed down a solid production cycle, we know what we are going to be spending in minerals every few weeks.
A large portion of capital went into the start of our Heavy Industry branch as seen by the grand change in expenses between November and December.
Two periods of expansion can be seen in December and February’s relatively small growth numbers as during these months we poured more capital into Carrier and Dreadnought BPOs, reducing final growth numbers. Also in February a large amount of minerals were purchased for another production cycle that extends into March. Since we operate on a two week cycle, materials purchased at the end of the month show up on the books in the next month.
Historical profits clearly show when the operation started to run.
In January and February we acquired three researched Dreadnought blueprints, two with perfect ME research. Our production lines have expanded to include these ship hulls and in March we should start seeing profits from these jobs.
*rabble rabble rabble*
*throws fuel into the fire*
*rabble rabble rabble*
Having the Tech 2 variant of the BPO means I run a serial production operation (1 BPO) at high margin with a lack of ability to change items in reaction to market changes.
The Tech 1 invention process is a parallel operation (1 or more BPOs) involving lower margins that can over saturate a Tech 2 producer. Inventors do need to maintain of a POS for ME/PE/Copy slots, but the ability to change items in response to market shifts is highly beneficial for Inventors.
Here is a quick rundown of the performance of my Tech 2 BPO vs inventing from one copy of the Tech 1 BPO.
If it was such a poor investment, then why get one? For me the lack of clicks in the invention process and the removal of the wretched POS in the production equation is worth the cost. I have stayed away from large-scale invention specifically due to these two reasons.
No major expansion desires are on the plate at the moment. Right now we are in a ‘slow and steady wins the race’ mode.