Aspnet core Parallels operation DbContext problem

Question

I've a problem when I use Pararrel function in aspnetcore, in particular when in the cycle i try to save something in database. I get my data from externarl api and deseserialize it in my class. This is the Parallel code.

Root players = JsonConvert.DeserializeObject<Root>(responseStream);
var bulkhead = Policy.BulkheadAsync(10, Int32.MaxValue);
var tasks = new List<Task>();
foreach (var player in players.players)
                    {

                        var t = bulkhead.ExecuteAsync(async () =>
                        {
            
                            int wyId = Convert.ToInt32(player.wyId);
                            HttpRequestMessage secondRequest = createRequest("https://apirest.com/v2/players/" + wyId + "?details=currentTeam&imageDataURL=true");
                            var client2 = _clientFactory.CreateClient();
                            var response2 = await client2.SendAsync(secondRequest);
                            if (response2.IsSuccessStatusCode)
                            {
                                var responseStream2 = await response2.Content.ReadAsStringAsync();
                                dynamic playerFullDetails = JsonConvert.DeserializeObject<dynamic>(responseStream2);
                                int wyId2 = Convert.ToInt32(playerFullDetails.wyId);
                                int marketValue = 0;
                                HttpRequestMessage tirthRequest = createRequest("https://apirest.com/v2/players/" + wyId2 + "/marketvalue");
                                var client3 = _clientFactory.CreateClient();
                                var response3 = await client3.SendAsync(tirthRequest);
                                if (response3.IsSuccessStatusCode)
                                {
                                    var responseStream3 = await response3.Content.ReadAsStringAsync();
                                    dynamic marketValueResponse = JsonConvert.DeserializeObject<dynamic>(responseStream3);
                                    if (marketValueResponse.marketValue != 0)
                                    {
                                        marketValue = Convert.ToInt32(marketValueResponse.marketValue);
                                    }
                                }
                                DateTime birthday = Convert.ToDateTime(playerFullDetails.birthDate);
                                int age = DateTime.Now.Year - birthday.Year;
                                Player finalPlayer = new Player();
                                finalPlayer.PlayerId = wyId2;
                                finalPlayer.MarketValue = marketValue;
                                finalPlayer.Value = Convert.ToDouble(marketValue) / Convert.ToDouble(1000000);
                                finalPlayer.Firstname = playerFullDetails.firstName;
                                finalPlayer.Lastname = playerFullDetails.lastName;
                                finalPlayer.Name = playerFullDetails.shortName;
                                finalPlayer.Position = playerFullDetails.role.name;
                                finalPlayer.Height = playerFullDetails.height;
                                finalPlayer.Foot = playerFullDetails.foot;
                                finalPlayer.IsLocked = false;
                                finalPlayer.Team = playerFullDetails.currentTeam != null ? playerFullDetails.currentTeam.name : "";
                                finalPlayer.TeamId = playerFullDetails.currentTeam != null ? playerFullDetails.currentTeam.wyId : 0;
                                finalPlayer.CompetitionId = 524;
                                finalPlayer.UpdatedDay = DateTime.Now;
                                finalPlayer.League = "Serie A";
                                finalPlayer.Age = age;

                                Player playerExist = await _context.Player.Where(x => x.PlayerId == wyId2).SingleOrDefaultAsync();

                                if (playerExist == null)
                                {
                                    if (finalPlayer.TeamId != 0)
                                    {
                                        await _context.Player.AddAsync(finalPlayer);
                                        await _context.SaveChangesAsync();
                                    }
                                }

                                if (finalPlayer.TeamId != 0)
                                {
                                    Team teamExist = await _context.Team.Where(x => x.TeamId == finalPlayer.TeamId).SingleOrDefaultAsync();
                                    if (teamExist == null)
                                    {
                                        Team team = new Team();
                                        team.TeamId = finalPlayer.TeamId;
                                        team.TeamName = finalPlayer.Team;
                                        await _context.Team.AddAsync(team);
                                        await _context.SaveChangesAsync();
                                    }
                                }
                            }
                        });
                        tasks.Add(t);
                    }
                    await Task.WhenAll(tasks);

The function isert 50/60 (in total would be 500) element in db and finally i receive this error

A second operation was started on this context before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913.

Thanks

Why do you create a MyseDbContext m and never use it? What I would do: * Query the REST API parallel. * Stop parallel work here * Load all existing and relevant ids from the DB at once * LOOP: If id does not exist: add new Player * SaveChanges — Klamsi, Apr 12 '21 at 06:38
Trying to run multiple bad queries concurrently only *increases* blocking and delays. 500 rows is *no data at all*. Besides, DB access is IO and `Parallel` is only meant for CPU-bound work. Using `Parallel` will block all cores on the server just to wait for a remote database server to respond. — Panagiotis Kanavos, Apr 12 '21 at 06:54
I agree. Async and parallel on resp api, Ok. Async and parallel on db insertion, not ok. you will just burn your finger. Bluck insert into db and ask the db to merge. Don't check if it exist or not in your code. Imagine db as building on the other side of the street. Would you cross the street and walk to this builing once with everything in a box. or will you do 500*3 crossing your self? And hiring a team of parallel atletes wont help. — Self, Apr 12 '21 at 06:54
I will also drop all `dynamic playerFullDetails = JsonConvert.DeserializeObject(responseStream2);`.. Dynamic only mean. I don't have time to copy past the class representing the data compile time will have to figure out. [How to auto-generate a C# class file from a JSON string](https://stackoverflow.com/questions/21611674/). As most of your code is deserilization and mapping property it will be done buy simply writing the class in one line. — Self, Apr 12 '21 at 07:02
For starters, the code needs rewriting. It failed with a tiny amount of rows, so it won't work with more data. The same veeeeeeery long method is trying to do everything - call a remote API *and* write the results to the database. *And* use a specific policy. *And* do all that per player. Split this into proper methods each of which does *one* thing properly. Once you do that you'll be able to use the *correct* structure and classes. Separate the HTTP API calls from the data code. Perform the API calls concurrently (ie without using Parallel), collect results and pass them to a data method — Panagiotis Kanavos, Apr 12 '21 at 07:02

score 0 · Answer 1 · answered Apr 12 '21 at 06:12

0

It's best practice to use 1 dbcontext per unit of work, and the dbcontext is not thread safe

So either create a new dbcontext per thread or lock access with someting like a semaphore otherwise you will always get unstable code

answered Apr 12 '21 at 06:12

Speedydown

91
4

ok thanks, How do I generate a new dbcontext? ive try using (var db = new MyDbContext()) but not work – Stefano Toppi Apr 12 '21 at 06:41
@StefanoToppi it does work. What doesn't work is trying to speed up bad data access by doing more of it. BTW a DbContext *is* a Unit of work already. It caches all changes and persists *all* of them in a single transaction when you call `SaveChangesAsync` – Panagiotis Kanavos Apr 12 '21 at 06:50

Aspnet core Parallels operation DbContext problem

1 Answers1