You are right that custom MP is the right way to do an integration of the custom/third-party monitoring system with SCOM. You have to think about three important things when you are planning your work on such MP:
- How you are going to get information from external system
- How you are going to persist and use it in SCOM
- How you are going to visualize it in SCOM
Let's walk through these three items:
From your intro it looks obvious - your system exposes RESTful API. SCOM (even 2012 or 2016) doesn't have native datasources to parse JSON so you'll need to create custom datasources using Powershell or C# (depends on your experience) . In this case, it might be reasonable to use any standard library to make this job easier.
SCOM has its special object model. You have classes to represent objects, monitors to detect failures/state changes and rules to collect performance metrics and alerts/events. So you'll need to implement Discovery datasources to get data about objects, monitored by your custom monitoring system (such as servers, databases, disks, apps, etc.) and define a class hierarchy to persist these objects in SCOM.
Then you'll need to create datasources for monitors and rules and here you must think before act - what failures, alerts and metrics you want to expose to SCOM. When you have clear understanding of this area - you are good to implement that (again - using PS or C#).
SCOM will give you some OOB visualization after you dome (1) and (2), so in the minimal scenario you'll need to define just a couple of views to show in SCOM console data collected by your MP. In ultimate case - if you want to have some fancy visualization - you'll have to create custom Dashboard. A good option here - use dashboards from SQL Server MP (it was released recently, it's free and it is really cool).
In fact, SCOM is not a monitoring system, but a framework, which has runtime platform, development language, and libraries, so building your own MP is closer to programming than IT administration :)
You also can try to use Silect MP authoring tool, but I'm not sure if it will help you to build custom datasources better than VS.
Good luck!
P.S. feel free to ping me via LinkedIn for more details about MP development.