1

I am looking for a way to generate random data for testing purposes (around 10000 files). For this testing I have a json schema.

Generation JSchema:

RunTest(){
  JSchemaGenerator generator = new JSchemaGenerator();
  JSchema schemaBuilding = generator.Generate(typeof(TestClass));
 }

The code itself is in C# so ideally I would have a C# code for doing this, though a python solution is also accaptable. I have found a number of questions on this topic that got websites or only focusses on a single prefixed sample but I can't find how to do this in C# (or less preferably python), anybody got any good way of doing this?

As for the reason for doing this: it's two fold: 1 this tests the stability of the system by entering a lot of random data looking for edge cases we haven't thought of and 2 it's a load test. (so basically a smoke+load test)

Dharman
  • 30,962
  • 25
  • 85
  • 135
Thijser
  • 2,625
  • 1
  • 36
  • 71
  • If anyone has any questions or suggestions that aren't answers, please feel free to post a comment. – Thijser Dec 17 '20 at 09:03
  • 1
    Think about it for a second. If you're generating JSON which you know fits the JSON Schema, are you really testing the JSON Schema? You'd be better off generating JSON by hand based on what you THINK the schema should do. That way you can actually test it, rather than have false confidence in tests which really only will fail if the schema is changed. – Relequestual Dec 17 '20 at 09:23
  • @Relequestual the schema itself is generated using a static code analysis (and is dynamic), and there is a rather large code base hiding behind the schema, I;m not really interested in testing the parses but rather all the code hiding behind the parser. I know that manually crafting the JSons is better but given the size of the entire thing that's not very practical at this moment (time limit) and running select tests for known problem areas and a large amount of random jsons (looking at logs when it crashes) will hopefully offer some protection. – Thijser Dec 17 '20 at 09:27
  • Note that I do agree that it would be better to manually craft them, however given the nested nature of the Jsons and some of them kicking off processes with all kinds of dependencies and a time limit on development time I sadly don't think I will be able to get the thousands of jsons needed(some nested processes might just kick of other things in various situations) plus I lack the knowledge as the code was externally developed to properly evaluate what jsons are needed. – Thijser Dec 17 '20 at 09:32
  • 1
    I can appreciate all the above. Even if you successfully manage to do what you want, it's only going to give you a false sense of security. In reality, all you'll be testing is that the JSON generation worked correctly according to the schemas. Generating JSON based on schemas is a good starting point for other things. Using them directly for tests against the schemas is like testing 1 === 1. – Relequestual Dec 17 '20 at 09:36
  • @Relequestual I undstand that, the idea however is that in principle the Schema generated by the code (combination of the schema+annotation) should at least describe (be used to check) incoming Jsons, any json that meets the schema is then send to be processed, we have tests for that. The problem is that following this step we have previous seen sudden massive delays as something parts of the program don't scale well, I'm hoping that by flooding the system with fake jsons we can smoke out the parts that fail to scale (for example turned out that a certain previous piece of code would loop ... – Thijser Dec 17 '20 at 09:42
  • ... the maximum size of integer times if a certain value was null, given the server size that took 5 minutes, not a crash but should be visible in a stress test if that happens elsewhere) – Thijser Dec 17 '20 at 09:42
  • Right, yes, for smoke testing, that's a potentially viable idea. Sadly I can't suggest anything to you in C#. We only list one tool on our site https://json-schema.org/implementations.html#data-from-schemas, plus there is "JSON Schema Faker" which some use. I've not personally tried either. – Relequestual Dec 17 '20 at 09:49
  • 2
    Since you have the class maybe you this will be useful https://github.com/nickdodd79/AutoBogus that will give you an object and then you can serialize it to json.... – jjchiw Dec 17 '20 at 11:58
  • @jjchiw thank you, autobogus and serializing the answer was exactly what I was looking for – Thijser Dec 17 '20 at 13:20

1 Answers1

1

In Oxygen Developer there is a tool that allows you to generate random JSON files from a JSON Schema, but you need to do this manually from an interface. The action can be found in the Tools menu and it opens a dialog box where you can configure various options for generating the JSON instances.

You can find more details in the user manual: https://www.oxygenxml.com/doc/versions/23.0/ug-editor/topics/json-schema-instance-generator-2.html

Octavian
  • 416
  • 2
  • 4