0

I've recently written a program which helps in the creation of maps for a game. In essence, the game's map is divided into multiple different "provinces", which are defined in the game's "provinces.bmp" file as pixels of a single colour grouped together.

With my program, there are two/three bits of code that use or manipulate PIL's Image object, to be displayed by tkinter:

  1. During the loading stage of the program, I need to store each pixel in it's corresponding "Province" object, sorted by colour. This is currently done by iterating through every pixel, which hurts my soul (The map in the base game is in the order of 5000x2000 pixels, for a total of 10 MILLION iterations):
def populatePixels(self):
    print("Populating Pixels... This may take a while")
    sys.stdout.flush()
    for y in range(0, len(self.provinceMapArray)):
        for x in range(0, len(self.provinceMapArray[y])):
            self.getProvinceAtIndex(x, y).pixels.append((x, y))

def getProvinceAtIndex(self, x, y):
    province = self.colorsToProvinces.get((self.provinceMapArray[y][x][0], self.provinceMapArray[y][x][1], self.provinceMapArray[y][x][2]))
    return province
  1. Users can switch the "MapMode" of the map, seeing it coloured not by each province's base colour, but instead by specific pieces of information about the province, such as Culture or Owner. To generate the Images for each MapMode, I iterate through every province, and every pixel in that province, and manually colour those pixels based on the corresponding field of the given mapmode.
def generateImage(self):
    self.image = Image.new("RGB", self.model.provinceMapImage.size)
    for province in self.model.idsToProvinces:
        if province == None:
            continue
        self.updateProvince(province)

def updateProvince(self, province):
    if self.image:
        fieldValue = province.getFieldFromString(self.name)
        if not (fieldValue in self.colorMapping):
            self.colorMapping[fieldValue] = (randint(0, 255), randint(0, 255), randint(0, 255))
        for pixel in province.pixels:
            self.image.putpixel(pixel, self.colorMapping[fieldValue])
  1. Data in each province can change by user input, and whenever this happens, the corresponding mapmode needs to be updated. Thus, we need to iterate through every pixel in the province and colour in the new field. In this case, "updateProvince" in the snippet above is used, and then tkinter is told to re-load the updated image.

My question is now: how should I best optimize this code? These are the three chunks that are slowing everything down, and I'm not sure how to best go about speeding everything up. Ideally since everything follows the pattern of needing to "iterate over every pixel", I initially considered python's Cuda Library, but I'm worried that in the first case, it wouldn't be able to handle all the different data structures, and in the second case it would be difficult to set up since each set of pixels is stored inside province objects, of which there are many.

Thank you all for your consideration. If there is still any confusion about my code, I will gladly clarify.

EDIT: here is a Minimal Reproducable Example, as requested

from random import randint
import tkinter as tk
from PIL import Image
import numpy
from PIL import ImageTk
from sys import stdout

class Province:
    def __init__(self, id, colour, resource):
        self.id = id
        self.colour = colour
        self.pixels = []
        self.resource = resource

def updateProvinceOnMap(mapImage, province, resourcesToColour):
    colour = resourcesToColour[province.resource]
    for pixel in province.pixels:
        mapImage.putpixel(pixel, colour)

if __name__ == "__main__":
    colourToProvinces = dict()
    provinces = []
    provinceMapImage = Image.open("../TestMod/map/provinces.bmp")
    provinceMapArray = numpy.array(provinceMapImage)
    resourcesToColour = {"Livestock": (0, 255, 0), "Iron": (96, 96, 96), "Gold": (255, 255, 0), "Wood": (102, 51, 0)}
    resources = ["Livestock", "Iron", "Gold", "Wood"]
    resourceCount = 4
    idCounter = 1

    # Issue 1: Populating the provinces from the image
    print("Populating Provinces")
    stdout.flush()
    for y in range(0, len(provinceMapArray)):
        for x in range(0, len(provinceMapArray[y])):
            colour = (provinceMapArray[x][y][0], provinceMapArray[x][y][1], provinceMapArray[x][y][2])
            if colour not in colourToProvinces:
                resource = resources[randint(0, 3)]
                newProvince = Province(idCounter, colour, resource)
                idCounter += 1
                colourToProvinces[colour] = newProvince
                provinces.append(newProvince)
            colourToProvinces[colour].pixels.append((x, y))

    # Issue 2: Creating image from a province's field; in this case, resource
    print("Generating Map")
    stdout.flush()
    mapImage = Image.new("RGB", (len(provinceMapArray), len(provinceMapArray[0])))
    for province in provinces:
        updateProvinceOnMap(mapImage, province, resourcesToColour)
    
    window = tk.Tk()
    window.geometry("1024x776")
    canvas = tk.Canvas(window, width=len(provinceMapArray), height=len(provinceMapArray[0]))
    canvas.pack()
    canvas.create_image(0, 0, image=ImageTk.PhotoImage(mapImage))
    window.mainloop()

As for the image used, The Base Game's Province.bmp Image is a good working example

  • Can you please provide a [mre]? It will be easier to test possible solutions/see all of the data structures. Also if you are going to iterate over come code 10 million times, try writing that piece of code in another programming language. Also please remove the `tkinter` tag. – TheLizzard Sep 07 '22 at 22:58
  • I can, but I also have the entire program on github too if that works. If not, I can whip up a small program that has the same code snippets. Also, the issue I have with iterating over a piece of code 10 million times in another language is that, the code is meant to populate local data structures, and I'm not sure if that can carry over well from another language. – Graapefruit Sep 07 '22 at 23:03
  • We need a small working example that we can run and reproduce the problem, otherwise we might make wrong assumptions about your code. – TheLizzard Sep 07 '22 at 23:05
  • Will do, I'll whip something together. Thank you for your help thus far. I'll let you know once its updated. – Graapefruit Sep 07 '22 at 23:18
  • It might take rethinking some of the ways you're doing things, but I would convert the image to a numpy array and do as much of the work as you can using numpy functions, which can be extremely fast. If you want to get even faster you can use cupy which is a replacement for numpy that uses the GPU. – Colin Sep 07 '22 at 23:37
  • First things first, you need to use Numpy for ND arrays. Then, you need to vectorize functions as much as possible. It will be hard for some function regarding the provided code. You can then use Cython or Numba so to write fast loop-based code operating on Numpy arrays. Then you can parallelize the code if this is not enough (using the multithreading embedded in Cython/Numba for example). Only then, you can think of using the GPU if this is still not enough (it will certainly be) since using efficiently a GPU is not so easy and requires the previous steps to be done anyway. – Jérôme Richard Sep 08 '22 at 00:03
  • Thank you all for your comments. I'll start modifying my code to use Numpy and Vectorize things. Regardless, the MRE has been editted-in the question's main body – Graapefruit Sep 08 '22 at 00:35
  • I don't understand much of what you are trying to do, but if you have under 256 colours you might be able to leverage the palette to change colours very quickly- see https://stackoverflow.com/a/52307690/2836621 Also, I have no idea why anyone would use a BMP file now that we have lossless, compressible, efficient, standardised PNG files. – Mark Setchell Sep 08 '22 at 00:52

0 Answers0