Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a completeness breakdown chart with benchmarks #72

Open
jreina opened this issue Oct 10, 2018 · 2 comments
Open

Create a completeness breakdown chart with benchmarks #72

jreina opened this issue Oct 10, 2018 · 2 comments

Comments

@jreina
Copy link
Member

jreina commented Oct 10, 2018

Create a chart comparing ShittyLINQ.js to System.Linq showing whether or not ShittyLINQ.js has a given method. If a method exists, provide a benchmark for the method in both assemblies showing the performance difference. Describe your methodologies.

@the-pat
Copy link
Contributor

the-pat commented Oct 3, 2021

I'll take this one.

@jreina and @jakehamtexas, what do y'all think about using GitHub actions to update the benchmarks? The Continuous Benchmark action seems pretty useful for this purpose. Thoughts?

@jakehamtexas
Copy link

jakehamtexas commented Oct 4, 2021

I like the idea of doing this in a CD context. But holy moly, this issue looks like it would scale pretty steeply with the size of the library increasing. For each method you would need -

  1. A benchmark test case defined
  2. A way to dynamically invoke the test case in both assemblies (presumably the same runtime)

The problems could really start cascading at this point. This is integrating some test surface areas that haven't been previously considered - especially fidelity to the LINQ function signature. If the function signature in both assemblies is not exactly the same, you could be stuck eating somebody else's bad implementation when you have to fix a compiler error to get your test case to run.

Maybe adding test cases would be better suited to their own issues, and creating a script/scaffold for adding the test cases and displaying them in doc would be a good standalone issue.

@the-pat @jreina

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants