spotmarket – pivot and resample

Overview

I published v0.2, which includes a bunch of hacked together proof of concept code. This week I dove into the pandas library, focusing on the data manipulation tools that are inherit to pandas. This post covers some of the highlights and demonstrates the initial charting that I’ve been producing.

More to come on what I plan to do with the data; for now those details have to be a trade secret.

Data Frame

The crux of pandas is the dataframe and this week I discoverd that you can load your query results right into a dataframe and not have to write a parser. Notice in the below code that I run the query and place the results into a dataframe with a ‘fetchall()’. This really saves me a lot of headaches when passing around chunks of data and makes it easy to write functions to combine data with overlapping values. Return the dataframe and do more ::work::.

Here’s a query that I use to get NPC kills by region when given a regionID.

def getnpckills_byregion(regionID):
    conn = psycopg2.connect(conn_string)
    cursor = conn.cursor()
    sql = """SELECT
      "mapSolarSystems"."regionID",
      "mapRegions"."regionName",
    SUM(mapkills."factionKills") as SUM_factionKills,
      mapkills."timestamp"
    FROM
      data.mapkills,
      public."mapRegions",
      public."mapSolarSystems"
    WHERE
      "mapRegions"."regionID" = "mapSolarSystems"."regionID" AND
      "mapSolarSystems"."solarSystemID" = mapkills."solarSystemID" AND
      public."mapSolarSystems"."regionID" = %s
    GROUP BY mapkills."timestamp", public."mapSolarSystems"."regionID", "mapRegions"."regionName"
    ORDER BY timestamp DESC
    """
    data = (regionID, )
    cursor.execute(sql, data)
    df = pd.DataFrame(cursor.fetchall(),columns=['regionID', 'regionName', 'SUM_factionKills', 'timestamp'])
    cursor.close()
    return df

Pivot

Doing a pivot in pandas is amazingly elegant and you can perform the operation in one statement.

2016-01-28_region_query_results

df.pivot(index='timestamp', columns='regionName', values='SUM_factionKills')

2016-01-28_region_query_results_pivot

Resample

Another powerful tool in the library are the resampling options. I’m still trying adjusting the parameters to find a balance on data granularity.

The below graphs show NPC kills over all regions over time. One graph has no resampling while the other has values that I feel display a daily value, normalizing the peaks and valleys that occur due to server downtime and differing activity rates due to timezone.

conversion = "12H"
v_window = 8
v_minperiods = 1
pd.rolling_mean(df.resample(conversion, fill_method='bfill'), window=v_window, min_periods=v_minperiods)

2016-01-28_universe_no_resample2016-01-28_universe_resample

Results

Ratting activity in Blood Raiders regions:

2016-01-28_blood_raiders_regions_sum 2016-01-28_blood_raiders_regions

Ratting activity in Serpentis regions:

2016-01-28_serpentis_regions_sum 2016-01-28_serpentis_regions

More to come. Find me on #k162space on tweetfleet.slack.com.

Advertisements

Sagain Kickout Station

The Sagain Lowsec system in the Tash-Murkon region is a major hub for jump capable ships as it allows you to reach Khanid, Querious, Stain, Catch, and the majority of Providence within one jump with Jump Drive Calibration V. Since Sagain is also a Lowsec station system that connects to contiguous Highsec, it is popular for Jump Freighter pilots as a midpoint connection into Highsec.

sagain_jump_range

The system has two stations that can be used for cyno traffic. Sagain VIII – Moon 11 – TransStellar Shipping Storage is a Gallente station that has excellent cyno placement opportunities while the other station, Sagain V – Inherent Implants Biotech Production, is much more problematic due to its shape and how Eve handles object barriers.

Sagain VIII - Moon 11 - TransStellar Shipping Storage

Sagain VIII – Moon 11 – TransStellar Shipping Storage

Sagain V - Inherent Implants Biotech Production

Sagain V – Inherent Implants Biotech Production

All objects in the Eve universe have a sphere around them that dictates their physical barrier. With the Gallente station being flatter and in a curve shape with a large section missing, you undock within the sphere barrier and can dock immediately. As shown here you are well within the station’s edge sphere when undocking.

TransStellar Shipping Storage Undock

TransStellar Shipping Storage Undock

The design of the Amarr station pushes you out the bottom and right out of 0 docking range — hence the ‘kickout’ nickname for stations of this type.

Even with my speedy shuttle that issued a stop command after undocking, I was at 1,000 m on the station after the space environment rendered and ultimately ended up 3,300 m after reaching a full stop.

Sagain V - Inherent Implants Biotech Production

Sagain V – Inherent Implants Biotech Production

Be careful when choosing a station for cyno activity by keeping the geometry of the station in mind. You don’t want to end up like these careless Jump Freighter pilot that I can only assume used the wrong station and got bumped out of docking range after undocking.

2/20 8 B
2/18 7.2 B
2/17 7.9 B
2/17 7.5 B
2/15 7.6 B
2/15 24.3 B


F2OY-X Gallente Administrative Outpost

Yesterday a new outpost went up in Querious in F2OY-X.

If you listen to the June 28 State of the Alliance (soundcloud.com) address from TEST leadership, you will learn that TEST originally requested that Goons not get involved in their ‘Good Fight’ campaign.

Now that things have escalated, Goons are on the field and are going to be entrenched in this campaign. Since a large numbers of Goon corporations need a place to call home, and a Gallente Office is the best Outpost choice since it contains the most number of offices.

Banlish on kugutsumen.com has been documenting Outpost builds and their political significance for a while now. Here is a recent quote from him on the reason this outpost went up and what it means for the war:

When I was living in Querious leading AtlasDOT and then into leading in FAIL I tried to push forward the idea to the AAA block that to control Delve/PB/Querious they should put office outposts in EVERY single system that connected to NPC delve. I was shot down time after time and no one would listen to me, being able to make every fleet that left 1DH or 319 run past an office outpost controlled by the different alliances in the SoCo would make life there a living hell… …

With F2O now having a station, only 4 other locations remain to put office outposts up and totally ruin the ability for the SoCo to pretty much ever take back the Delve core. … the outpost Goons put down is in SUCH a good place to fucking ruin everyone’s day. … Surrounding NPC 0.0 with office outposts is a good way to beat on the people that take refuge there…

source

In addition to this post, Banlish is a welth of political knowledge and he has many comments in the large, large thread called 0.0 Politics Discussion.

In January of 2010 during the purge of Cloud Ring, Goons established control by dropping a Gallente outpost in  B-DBYQ. It seems like this strategy works and so we are seeing it again with the Delve campaign.

Side Note: I’ve found some gems of history about other regions and Outposts. As I find them, l will link them here:
Branch History


Delve Financial Warfare

After I get home from work and eat dinner, I usually only have around an hour a night that I can dedicate to Eve. This open hour doesn’t give me a lot of time to participate in any of the large TEST fleets so I am going to help out the war in a different way. In addition to the spaceship war, I am going to launch a fiancial war on the current residents of Delve, Querious, and Period Basis.

One of my trading alts is going to fly around and mess up the markets. I’m going to polish off his skills so I can have region-wide abilities to buy, sell, and modify prices.

Here are some rough goals:

  1. Drive up the prices of doctrine items
  2. Buy out stocks of ships, ammo, guns, and fuel
  3. Undercut to drive out logistical people that stock markets

I’m going to start with 31B in funds and possibly scale up to 50B depending on how successful this becomes.

I have never done any type of mass manipulation on this scale so there is a high risk that it will not be successful. I will try to document any significant advancements in my campaign here so stay tuned.

#delve2012