0

I am writing a code to upload users timesheet to AWS S3 bucket. There are around 150 users, so I am trying to make the code to run concurrently for all the users using waitgroup. Below is my main() which is working without errors, but it is uploading incorrect data. ie. uploading the same data for multiple users in a span. I believe the same variables are getting updated for all the users. Can anyone point me how to solve this issue?

 func main() {
    // Get list of users
    agentsList := getAgentsList(listUsersURL, 1)
    fmt.Println("Total Users:", len(agentsList))

    // Get call details of all the users
    var waitgroup sync.WaitGroup
    waitgroup.Add(len(agentsList))

    for count, user := range agentsList {
        go func() {
            fmt.Println("\nAdding User in waitgroup", count)

            FirstCall, LastCall := getFirstAndLastCall(user.UserID)

            // Upload timesheet to S3 bucket
            postTimeSheetToS3(timesheetDate, user.UserID, FirstCall, LastCall)

            }
            fmt.Println("\nRemoving User from waitgroup", count)
            waitgroup.Done()
        }()

    }
    waitgroup.Wait()
}
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Rahul Satal
  • 2,107
  • 3
  • 32
  • 53
  • 1
    No, no're not getting an incorrect result from the goroutine. You're getting an incorrect result from the loop. `user` is re-used for each iteration of the loop. This means that frequently, `user` is changed before `UserID := user.UserID` is executed in your goroutine. One solution is to move that line before the `go func() {` line. – Jonathan Hall Aug 31 '19 at 07:59
  • Yes, but only after adding waitgroup in it. Otherwise, I was getting the correct result with the loop. – Rahul Satal Aug 31 '19 at 08:01
  • Previously, the code was working as expected. but it was taking around 2 mins to complete. Therefore, I have added waitgroup and now I am not getting correct result. However, I can try moving UserID := user.UserID before the go func() line. – Rahul Satal Aug 31 '19 at 08:10
  • [With waitgroup](https://play.golang.org/p/L-whYYh4k6Q) and [without waitgroup](https://play.golang.org/p/Y9NfPGwhlti) have the exact same problem. – Jonathan Hall Aug 31 '19 at 08:10
  • 1
    You are right, moving the line before func() worked. The result was exactly the same as your last comment - the very last element duplicated many times. Thanks for giving me the solution. – Rahul Satal Aug 31 '19 at 08:24
  • Great! Glad to hear it :) Note you have the same issue with the `count` variable, but it's less serious, since it's only used for textual output. If you want to fix that, try this putting this before `go func()`: `c := count`; then use `c` within your `Println` statements. – Jonathan Hall Aug 31 '19 at 08:25
  • 1
    oh I see. That is also fixed. Thanks again. Really you have saved my lot of time. – Rahul Satal Aug 31 '19 at 08:33

0 Answers0