the metrics website is live. https://asyncapi-metrics-dashboard.vercel.app/dashboard/DeveloperExperience/1W

PR is ready for review. https://github.com/Amzani/asyncapi-metrics-dashboard/pull/6

Started collecting some test metrics in CLI to build the dashboard.

We have an initial UI for the dashboard @KhudaDad414 will continue the work on this

Decided to go for Tremor library instead of default NewRelic widgets:

• We will have more flexibility on how to show and compute insights (e.g Time to First API design is impossible to show in NR dashboard https://github.com/Amzani/asyncapi-metrics-dashboard/issues/2#issuecomment-1946098809)
• As a visitor, you need a NewRelic account to see NewRelic dashboards
• Future opportunity to centralize all our Insights in one place

Rabbit holes:

• To prevent being Rate Limited we can cache NR API Requests

System errors

SELECT sum(asyncapi_adoption.action.finished) as 'System errors'
FROM Metric
WHERE success = false

Validation errors

SELECT sum(asyncapi_adoption.action.finished) as 'Validation Errors'
FROM Metric
WHERE action = 'validate'
AND validation_result='invalid'

Time to fix a validation error

SELECT filter(latest(endTimestamp), WHERE action = 'validate' AND validation_result ='valid' AND source is not null AND success = true)/1000 - filter(latest(endTimestamp), WHERE action = 'validate' AND validation_result ='invalid' AND source is not null AND success = true)/1000 AS 'TS'
FROM Metric FACET source, user

Then compute averages, median...

For Time to first API Design we expect to add two metadata in newRelic Metric table :

• user -> UUID v4
• source -> SHA1(FileName)

e.g

SELECT filter(latest(endTimestamp), WHERE action = 'generate:fromTemplate' AND source is not null AND success = true)/1000 - filter(latest(endTimestamp), WHERE action = 'new:file' AND source is not null AND success = true)/1000 AS 'TS'
FROM Metric FACET source, user

Then compute averages, median...

SELECT count(1) doesn't work as expected.

SELECT sum(asyncapi_adoption.action.finished) as 'Asyncapi V3 adoption'
FROM Metric 
WHERE action = 'validate' 
AND  asyncapi_version = '3.0.0'

SELECT sum(asyncapi_adoption.action.finished) as 'Created AsyncAPI files'
FROM Metric 
WHERE action = 'new:file'

Dashboard: https://onenr.io/0Vwga07k8jJ

We have Number of created AsyncAPI files and AsyncAPI 3.0x Adoption

In NewRelic

AsyncAPI 3.0x Adoption

SELECT count(1) FROM Metric WHERE metricName = 'asyncapi_adoption.action.finished' AND action = 'validate' AND  asyncapi_version = '3.0.0' SINCE last week

Number of created AsyncAPI files

SELECT count(1) FROM Metric WHERE metricName = 'asyncapi_adoption.action.finished' AND action = 'new' SINCE last week

Checking the metrics we collect