I am trying to optimize my code to run faster if anybody has a solution? It currently takes around 10-15 minutes to run

Question

(Using Python) My code reads through a text file and biases the point to make a perfect circle. When it is reading through the points, it is taking 10-15 minutes to run. I have multiple different text files of similar size to go through so waiting that long is impractical. Does anybody know of ways I can edit my code to make it run faster? It has to read through roughly 430 points each file.

import numpy as np
import math
import matplotlib.pyplot as plt
from circle_fitting_3d import Circle3D

#Read the text file with all the points
lines = open(text file goes here)

#create a 2D array of all the (x,y,z) points in the file
points = np.array([[float(n) for n in ln.split(",")] for ln in lines])

#find the center position (x,y,z) of all the points in the file
center = points.mean (axis = 0)

#find circle of best fit
circle_3d = Circle3D(points)
print(circle_3d._projected_points)

#calculate all the distances between the points and the center and get the mean
#print("Radius = " + str(circle_3d.radius) + "mm")

#move the circle center to (0,0,0)
translateTuple = circle_3d.center
print("Translation Values: " + str(translateTuple))
transPoint = np.empty((0,3))
for p in points:
    tx = p[0] - translateTuple[0]
    ty = p[1] - translateTuple[1]
    tz = p[2] - translateTuple[2]

#print("Diff x: " + str(tx-p[0]))
    transPoint = np.append(transPoint,[[tx,ty,tz]], axis=0)

#Calculating new points
def pointOnLine(pointA, pointB, dis):
    h = math.sqrt((pointA[0]-pointB[0])**2 + (pointA[1]-pointB[1])**2 + (pointA[2]-pointB[2])**2)
    cx = pointA[0] - ((dis*(pointA[0]-pointB[0])/h))
    cy = pointA[1] - ((dis*(pointA[1]-pointB[1])/h))
    cz = pointA[2] - ((dis*(pointA[2]-pointB[2])/h))
    pointC = np.array([cx,cy,cz])
    return pointC

#print(transPoint)
new_circle = Circle3D(transPoint)

#resize the circle
resizeValue = new_circle.radius - 100.00
print(resizeValue)
resizePoints = np.empty((0,3))
for p in transPoint:
    resized = pointOnLine(p, new_circle.center, resizeValue)
    resizePoints = np.append(resizePoints,[resized], axis=0)

#print(resizePoints)
resizePoints = Circle3D
resize_circle = Circle3D(resizePoints)

print("Old Radius = " + str(circle_3d.radius) + "mm")
print("New Radius = " + str(new_circle.radius) + "mm")
print("Resized Radius = " + str(resize_circle.radius) + "mm")
print(circle_3d.center)
print(new_circle.center)
print(resize_circle.center)

#write new points to .txt file
with open('poly_out_test_new_points.txt', 'w') as testfile:
    for row in reversed(resizePoints._projected_points):
        testfile.write(' '.join([str(a) for a in row]) + '\n')

#plot in 3D space
fig = plt.figure()

# syntax for 3-D projection
ax = plt.axes(projection ='3d')
ax.plot3D(points[:,0], points[:,1], points[:,2], 'green')
ax.plot3D(circle_3d._projected_points[:,0], circle_3d._projected_points[:,1], circle_3d._projected_points[:,2], 'purple')
resize_circle.plot(ax)

#circle_3d.plot(ax)
plt.show()

print("Complete")`

I have tried using array broadcasting, but this is my first time using that, so it kept throwing me errors and I wasn't sure where I was going wrong. The code I posted is the solution I have that works but runs slow.

Have you already checked which part of the code is your bottleneck, i.e. takes the longest? — FlyingTeller, Jun 16 '23 at 14:02
the section that takes the longest is the resize math so where the comment says "Calculating new points" through the comment that says "resize the circle". Up to there takes a couple seconds and once it gets through that the code also finishes in a couple seconds. — Artemis Pelle, Jun 16 '23 at 14:05
circle_fitting_3d is a library I used to help create a circle of best fit on 3d points. I used it to bias the old points to create the new circle — Artemis Pelle, Jun 16 '23 at 14:06
`np.append` is pretty slow when called repeatedly (it copies the whole cumulative array each time). Maybe create a list of the arrays returned by `pointOnLine`, and then combine them all in one step with `np.vstack` ? — slothrop, Jun 16 '23 at 14:09
If you make a list of arrays, another option instead of `np.vstack` is `np.reshape`, as in https://stackoverflow.com/a/7134033/765091 — slothrop, Jun 16 '23 at 14:15
Also, check the line of code `resizePoints = Circle3D`. This looks like a typo since it overwrites your `resizePoints` array and replaces it with a reference to a class. — slothrop, Jun 16 '23 at 14:18
If the bottleneck is in the file reading, have you tried to use points = np.loadtxt("your_file", delimiter=',') to get an array directly ? — matleg, Jun 16 '23 at 14:32

AirSquid · Answer 1 · 2023-06-16T16:41:26.347

Yer not gonna believe the bottleneck...

In your loop when you are resizing:

for p in transPoint:
    resized = pointOnLine(p, new_circle.center, resizeValue)
    resizePoints = np.append(resizePoints,[resized], axis=0)

The library you are using appears to be recalculating the circle center in every iteration (in your case 430 times). This is expensive. And odd as it looks like a property rather than a method/function call, but I didn't go spelunking to look at the source code.

change your loop to:

ctr = new_circle.center
for p in transPoint:
    resized = pointOnLine(p, ctr, resizeValue)
    resizePoints = np.append(resizePoints,[resized], axis=0)

When I was testing with 400 points, that change reduced the time for that segment of code from 160 seconds to 0.4 seconds.

The rest of the suggestions are good, but window dressing for a number of points this small.

Here's your code with my tinkering. Note I think you had some typos that I commented with ????

import numpy as np
import math
import matplotlib.pyplot as plt
from circle_fitting_3d import Circle3D
import time

#Read the text file with all the points
# lines = open(text file goes here)

#create a 2D array of all the (x,y,z) points in the file
# points = np.array([[float(n) for n in ln.split(",")] for ln in lines])

points = np.random.randint(0,1000,1200).reshape((400, 3))

# print(points[:10])

#find the center position (x,y,z) of all the points in the file
center = points.mean (axis = 0)

tic = time.time()
#find circle of best fit
circle_3d = Circle3D(points)
toc = time.time()
print(f'circle fit {toc - tic:0.2} sec')
# print(circle_3d._projected_points)

#calculate all the distances between the points and the center and get the mean
#print("Radius = " + str(circle_3d.radius) + "mm")

tic = time.time()
#move the circle center to (0,0,0)
translateTuple = circle_3d.center
print("Translation Values: " + str(translateTuple))
transPoint = np.empty((0,3))
for p in points:
    tx = p[0] - translateTuple[0]
    ty = p[1] - translateTuple[1]
    tz = p[2] - translateTuple[2]

#print("Diff x: " + str(tx-p[0]))
    transPoint = np.append(transPoint,[[tx,ty,tz]], axis=0)

toc = time.time()
print(f'translated in {toc - tic:0.2} sec')
#Calculating new points
def pointOnLine(pointA, pointB, dis):
    h = np.sqrt((pointA[0]-pointB[0])**2 + (pointA[1]-pointB[1])**2 + (pointA[2]-pointB[2])**2)
    cx = pointA[0] - ((dis*(pointA[0]-pointB[0])/h))
    cy = pointA[1] - ((dis*(pointA[1]-pointB[1])/h))
    cz = pointA[2] - ((dis*(pointA[2]-pointB[2])/h))
    pointC = np.array([cx,cy,cz])
    return pointC

#print(transPoint)
new_circle = Circle3D(transPoint)

#resize the circle
resizeValue = new_circle.radius - 100.00
print(resizeValue)
tic = time.time()
resizePoints = np.empty((0,3))
ctr = new_circle.center
for p in transPoint:
    resized = pointOnLine(p, ctr, resizeValue)
    resizePoints = np.append(resizePoints,[resized], axis=0)

toc = time.time()
print(f'resized in {toc - tic: 0.2f} sec')

#print(resizePoints)
# resizePoints = Circle3D    # ????  edited...
resize_circle = Circle3D(resizePoints)

print("Old Radius = " + str(circle_3d.radius) + "mm")
print("New Radius = " + str(new_circle.radius) + "mm")
print("Resized Radius = " + str(resize_circle.radius) + "mm")
print(circle_3d.center)
print(new_circle.center)
print(resize_circle.center)

#write new points to .txt file
with open('poly_out_test_new_points.txt', 'w') as testfile:
    for row in reversed(resize_circle._projected_points):         # ????  edited
        testfile.write(' '.join([str(a) for a in row]) + '\n')

#plot in 3D space
fig = plt.figure()

# syntax for 3-D projection
ax = plt.axes(projection ='3d')
ax.plot3D(points[:,0], points[:,1], points[:,2], 'green')
ax.plot3D(circle_3d._projected_points[:,0], circle_3d._projected_points[:,1], circle_3d._projected_points[:,2], 'purple')
resize_circle.plot(ax)

#circle_3d.plot(ax)
plt.show()

print("Complete")

Great spot! Yes, it looks like it's doing a somewhat heavy calculation without memoization - https://github.com/CristianoPizzamiglio/circle-fitting-3d/blob/32a72b323f09a8c730c39ba76d9a7bc7657fedd2/src/circle_fitting_3d/circle_3d.py#L101 — slothrop, Jun 16 '23 at 15:00

I am trying to optimize my code to run faster if anybody has a solution? It currently takes around 10-15 minutes to run

1 Answers1