spotmarket Graph Sample

What’s data without some graphs? Here is a small snip of the graphing that is currently in the spotmarket project.


2016-01-13 to 2016-06-07
17,802,640 Rows

2016-01-13 to 2016-06-07
9,819,421 Rows


Jumps in Trade Hubs


Jumps and NPC Activity in The Northern Regions


Gurista Ratting Rates


NPC Universe Overview


Outer Ring

First pass at a map using D3.js inspired by HelicityBoson at



spotmarket – 0.4


Whew, this was a big step in the right direction. I added a lot of sanity (keep everything in UTC), usability, and cleanup to the existing design. We’ve got moongoo, the beginnings of indexes, and CREST killmail data now flowing in to the growing dataset. The backend was improved by migrating to PostgreSQL 9.5.1 and I also cleaned up the install directions to try to make it easier.

0.4 Release

[bug] Change timestamp for zkillboard and markethistory consumer to use UTC for mental sanity
[enhancement] Add 404 error page
[enhancement] Change to postgres 9.5.1 to support jsonb
[enhancement] Parameterize graph functions
[enhancement] Change date format in graphs to ISO for mental sanity
[enhancement] Standardize table formatting
[enhancement] Supervisor to make Flask web service persistent
[enhancement] Add jsonb index on data.killmails for typeID and solarSystemID
[enhancement] Add paging to zKillboard consumer
[enhancement] Add check to resume from last recorded page for zKillboard consumer
[enhancement] Add basic exception handling to zKillboard consumer
[new] Creation of ship index report



CREST Verified Moon Minerals

It is no secret that I have an affinity for finding accurate moon data. In a previous post, I’ve gone so far as to chart the regional density of each moon mineral, showing that there were built in skews as to where each type was placed for each class (R8, R16, etc).

In the Rubicon expansion, CCP introduced siphon units, which can be anchored near a POS to slowly leech items from within the shields. When an extractor is destroyed the killmail will show what moon mineral was in it. This, combined with the x,y,z location data that started to be exposed after the Parallax expansion, can be used to locate the nearest celestial object to give us a verified report of what moon mineral is contained in the object.

I gathered some examples for anyalisis, and over a long flight, I started to put some code together to use killmail x,y,z coordinates to parse CREST killmails. After implementation and testing, it has proven to be accurate so I’ve included the feature in this release. There is a manual step to get the moon mineral data into a table where it is used in the web frontend; I wanted to keep them separate to isolate the two datasets.

PostgreSQL Upgrade to 9.5.1

Rather than writing a parser for CREST killmails, I decided to store the JSON itself for simplicity. Developing on 9.3 with the json datatype proved to be a path that I did not want to go down. 9.3 is not the prefered version and lacked support for a fancy datatype inherit to 9.4+ and above. The following blog post also convinced me to migrate.

The proof is in the numbers so here’s a side-by-side comparison of the same query on the same dataset with only the datatype being different. The two virtual machines also had access to the same amount of CPU and RAM to control the results.


This query was proof enough that I made the right choice. Later that night while checking my logging table, I also noticed an improvement in insert speed.

9.3 (json) [kills] insert 2982 @ 93.81 rec/sec
9.5.1 (jsonb) [kills] insert 2850 @ 124.44 rec/sec

Not bad for about 45 minutes of testing and correcting install instructions.

0.5 Release

What’s up for the next release? Check out the for a full list of items slated for each release. The main focus is going to be putting more control in the web front end, letting you enable/disable import items, add item to the market/zKillboard watch list, etc.

Rorqual Move

I spent a few hours moving a Rorqual to my new home system and almost got caught making a fatal mistake because of my lack of knowledge regarding standings if you are in Factional Warfare. It turns out that standings can affect where you dock when you are in FW.

Since this was my first capital operation since the Phoebe expansion, I wasn’t prepared for the amount of jumps Dotlan pulled it. I was thinking that it was going to be a simple 2-3 jump operation, but actually came out to 7.


Everything was going according to plan until I hit the Ofstold system, which is an Amarr system.

[ 2014.12.06 06:40:34 ] (info) You have been denied access for the following reason: The Amarr Empire denies access to Factional Warfare enemies. Either assist the Minmatar Republic to capture this system or retire from the war.

I paused for a second, let out an audible curse, and checked local — empty. The quickly formulated backup plan that came to mind was to warp to the sun, create a safe inline, and warp to the newly created safe. If someone came into local, I was going to log off and wait them out until I got my cyno character in position in the next system.


I was able to get my character in position in the next system, wait out the 12 minute fatigue timer, and safely land on the next station.


Even if you have been playing Eve for 5 years, there are still areas of the game that surprise me. I was quite lucky that the system that I could not dock in was currently empty.

Historical Profits by Solar System


CCP has released two out of the six Devblog posts aimed at industrialists detailing changes for the upcoming Summer expansion. We’re seeing sweeping changes to the way logistics are done for capital ships, station research, POS anchoring limitations, BPO security concerns, and how inventors are going to be given a boost with BPC copy rates. Lockefox has a great summary of the concerns in his Everything Is Changing post.

Given the ideology coming out of the Dev Blogs to empower Nullsec industrialists while kicking Lowsec in the knees, I wanted to see how much of my industrial gameplay occurs in Lowsec — Is CCP killing my game?

Data Data

Summary of profit grouped by solar system with profit numbers obfuscated.

— Get Profit per Station By Solar System including Region
SELECT SUM(profit), mapDenormalize.itemName, mapSolarSystems.solarSystemName, mapRegions.regionName, AVG(
FROM wallet
JOIN mapDenormalize
ON (wallet.stationID = mapDenormalize.itemID)
JOIN mapRegions
ON (mapDenormalize.regionID = mapRegions.regionID)
JOIN mapSolarSystems
ON (mapDenormalize.solarSystemID = mapSolarSystems.solarSystemID)
GROUP BY wallet.stationID
ORDER BY sum(profit) DESC


It turns out that Lowsec only accounts for 7% of our profits to date so I can’t complain about the nerf that is going to hit Lowsec capital builders given the upcoming compression changes.



The changes to compression are a welcomed change, even if it means retiring or heavily modifying the logistical chain for Lowsec capital production. I have a feeling that there are going to be more major changes in the next four upcoming Devblog posts. Stay tuned.

Increasing Faction Standings with Tags

For some time I have been queuing up production jobs that would be well suited for highsec. In order to anchor a POS in highsec, your corporation needs to have 5.0+ standings with the appropriate faction [detailed guide].

Since my main character has run a lot of Gallente missions in the past for ISK and has Social IV, he started off this task with a standing of 3.12 to the Gallente Federation.

Rather than grinding out more missions, I found out that there are a series of one-time agents for each faction that will trade in tags for standings. There are four Data Centers, one for each empire, located in New Eden where you can turn in tags.

You can collect these tags by running missions or by buying them off the market. A bunch of tags for one empire’s Data Center agents came out to around 352 M ISK on the Jita market. I didn’t see any major swings in prices or obvious market manipulation attempts, so I bought enough for all the agents.

Turning in these tags raised my standing from 3.12 to 4.95 with Social IV. After Social V completes, I should have more than the required 5.0 base standing in order to anchor a POS in highsec. [edit] Apparently Social skills do not come into play with anchoring rights. It appears I will have to do some more grinding to get the base standing up to 5.0+.

Chatting with all the Data Center agents

Since corporation standings are calculated as ‘the average of all corporation member standings towards that faction after the period of a week’, I will have to move this character into a 1 member corporation and wait for the standings to update.

No rush on this project, but I wanted to get the standings in order now so when it comes time to light up the production lines, I can immediately drop a POS. If you don’t want to raise your corporation’s standings yourself, you can even hire a guy to do it for you.

Perhaps I will wait for the POS rework before starting this production project. From what I have gathered listening to current CSM Chairman Seleene, the POS redesign is in the works. I haven’t been able to determine if this is slated for Winter 2012 or for Summer 2013.

Inferno Change List

Test builds are being deployed. I will attempt to keep a master list here so bookmark and refresh this post.

Patch Notes [link]
Patch Size: ~1.04 GB

Game of Drones:

Revamped the Inventory:
This affects:
– Managing corporation items in a station
– All item management from POS structures done in space on grid
– All Inventory management in space (wrecks, opening other ship locations, etc)
– All Inventory management in stations (items, ships)
– This change does NOT affect the ASSETS window

Revamped FW:
– LP for kills
– Bunkers are now infrastructure hubs
– LP for all FW actions (capturing complexes, capturing infrastructures hubs, etc)
– New FW front page
– Consequences for capturing systems/regions (benefits for you, grief for your enemy)
– Capturing FW system affect SOV ownership
– Map filters for FW have been revamped


• New missile launchers and launch effects
• New missiles, trails and explosions
• New Stealth Bombers
o Manticore
o Nemesis
o Purifier
o Hound
• Amarr V3
o All sub capital Amarr ships, excluding T3 cruisers.


“14 different skintypes have been added to the Character Creator. This allows all bloodlines to have a wide range of skincolors.
These skintypes replace the previous skintones that allowed players to modify their skincolor a little bit.
A character can keep his/her current skintone, but as soon as a skintype has been applied and the character saved, one of the new skintypes has to be used.

The skintypes can only be applied with full recustomization, that is when you also have the option to resculpt your character, but the plan is to give everyone an optional full recustomization when this goes on TQ.”

Super Friends:

• Updated Kill Reports (aka Kill Mails)
• Different war cost
• Corp CEO/Director declares wars, no voting required
• Alliance wars transfer to corps leaving
• No automatic war retraction by aggressor
• Wars last for week by default
• Either side can offer surrender in a war
• The defending side can get allies to help
• There is a place for people to find prospective allies/clients
• Players can now more easily track the war in the War Report
• All corps have publicly viewable war history
• Drone Damage module (Extrinsic Damage Amplifier)
• CPU Rigs (Small/Medium/Large Processor Overclocking Unit)
• Fueled Shield booster (Small/Medium/Large Ancillary Shield Booster), using Cap Boosters as charges
• Light and Medium Web drones
• Resistance shifting armor hardener (Armor Adaptive Hardener)


1. Frigate Rebalance [link]
Tormentor, Punisher, Merlin, Incursus, and Rifter changes.

2. New Modules
From Soundwave’s presentation at Fanfest, it sounded like the following modules are going to be included in Inferno. So far we only have this slide from FanFest and a long forum posts. [44:24]

Micro Jump Drive — Perhaps soon:

This list isn’t feature complete – more will come with time!
CCP Goliath link

Drone Damage module (Extrinsic Damage Amplifier)
CPU Rigs (Small/Medium/Large Processor Overclocking Unit)
Fueled Shield booster (Small/Medium/Large Ancillary Shield Booster), using Cap Boosters as charges
Light and Medium Web drones
Resistance shifting armor hardener (Armor Adaptive Hardener)

3. Missile Effects [link]
Youtube video starting at 3:30.

4. Missile Launchers [link]

5. Updated Kill Reports and War Reports
[initial concept devblog]
[updated devblog]

For a while, we’ve been talking about global killboards and all the awesome stuff that comes with it (leaderboards, weekly contests etc). Linking killmails in chat to other players means that you’ll effectively have access to that data, making our global killboard project slightly problematic. That, coupled with some security concerns means that we’ll have to withdraw linking killmails ingame.

Just to make it clear, we’re not pulling this feature because we don’t want you linking killmails, we’re pulling it because we have concerns about the impact on our global killboard project and making this massive amount of data available in a more graceful manner in terms of security etc.

It makes me sadface, but hopefully the tradeoff will be a better system down the road.
CCP Soundwave link

We fixed everything. You can now link killmails in chat! Thank Tuxford and Punkturis.
CCP Soundwave link

6. Unified Inventory UI [devblog]

7. Amarr v3 Ship Updates [preview youtube]


8. Updated Ship Models
Stealth Bombers
[unofficial renders on youtube]

9. Overview Fading [link] Included in Escalation

10. War Mechanic Updates [devblog]

11. FW Changes [devblog]

Without going into exhaustive detail:
1. Cannot dock in stations that are in systems controlled by an enemy
– Example: Minmatar cannot dock in stations in a FW system controlled by the Amarr/Caldari
– You also cannot use station services if you docked before system flipped
– Ninja Edit: This does not affect neutrals

2. There are 5 levels of discounts for a number of different things – if you own the system and have upgraded it
– Medical clone discounts (from 10% in increments of 10 up to 50% discount)
– Number of assembly lines increases by 1 for every level
– All broker fees discounted (from 10% in increments of 10 up to 50%)
– NOTE: You only get these discounts in the FW system
– ADDENDUM: These discounts apply to neutrals not in FW

3. Militia store LP prices are affected by faction wide performance (discounts in increments of 10%)
– Note that Datacores have been split and moved in FW LP stores
– Pending implementation, LP gain increase for all ways in which LP can be gained in FW – FW Kills, FW Missions, Complexs etc

I do believe these are the most prominent consequences implemented/pending.
CCP RubberBAND link

12. War Costs

SoniClover stated earlier in this thread that the war cost formula on Sisi now is really old, we’ve updated it since internally and the new version should be on Sisi soon.
CCP Punkturis link


1. Drake Rebalance

2. Titan Rebalance

3. Technetium Rebalance

Pushed Back

1. Destroyer and Battlecruiser Skill Rebalance

There will be no skill change for Inferno. Your destroyer and battlecruiser skills are safe for now. Battlecruisers will have to wait until we have rebalanced frigates and destroyers to have something clean to compare them with.
CCP Ytterbium link

2. Armor and Shield Rig Rebalance [link]

In conclusion, this is just too much controversial / poorly thought to be a first step to anything, we need to take a step back, breathe, and have a look at the whole passive versus active tanking situation through modules and ships as well as changing rigs before committing to more changes.
CCP Ytterbium link

Crucible Glamor Shots

The nebula rework that came in the Crucible expansion is very breathtaking. It ads such an amazing feel of location to the universe. Traveling from The Forge to Gallente space near Cloud Ring really gives me a sense of  locational awareness.

Another effect that has really impressed me is the miss animation of weapons. Fighting Amarr ships has never looked so good! Enjoy the 1680 x1028 goodness.