Add the ability to run tests against HTTP requests.

[cstick]

I really like this tool and wanted the ability to execute tests against http responses. This seems to relate to feature request #267.

[lgtm-com]

This pull request introduces 1 alert when merging ac9e236 into 0ea5e3e - view on LGTM.com

new alerts:

• 1 for Unused variable, import, function or class

[cstick]

Here are examples and their outputs.

GET https://cat-fact.herokuapp.com/facts/random HTTP/1.1

@tests
rc.test('status code is 200', () => {
    expect(response.statusCode).to.equal(200);
});

rc.test('content is not null', () => {
    expect(response.body).to.not.be.null;
});

var body = JSON.parse(response.body);

rc.test('type is cat', () => {
    expect(body.type).to.equal('cat');
});

###

GET https://cat-fact.herokuapp.com/facts/random HTTP/1.1

@tests
rc.test('status code is 200', () => {
    expect(response.statusCode).to.equal(200);
});

rc.test('content is not null', () => {
    expect(response.body).to.not.be.null;
});

var body = JSON.parse(response.body);

rc.test('text equals hello world', () => {
    expect(body.text).to.equal('hello world');
});

rc.test('type is cat', () => {
    expect(body.type).to.equal('cat');
});

###

GET https://cat-fact.herokuapp.com/facts/random HTTP/1.1

@tests
rc.test('status code is 400', () => {
    expect(response.statusCode).to.equal(400);
});

var body = JSON.parse(response.body);

rc.test('text equals hello world', () => {
    expect(body.text).to.equal('hello world');
});

###

GET https://cat-fact.herokuapp.com/facts/random HTTP/1.1

@tests
// asdf is not defined and will render test results that failed to execute.
asdf
throw "asd";

rc.test('status code is 200', () => {
    expect(response.statusCode).to.equal(200);
});

###

GET https://cat-fact.herokuapp.com/facts/random HTTP/1.1

Passing tests

Some passing tests and one failed with overall failure.

A JavaScript bug exists in test scripts.

Standard output without tests.

[lgtm-com]

This pull request introduces 1 alert when merging d8b185c into 0ea5e3e - view on LGTM.com

new alerts:

• 1 for Unused variable, import, function or class

[pke]

Very cool. Could this be run from the command line? I guess not, since it depends on VSCode?
The tests are written directly in the http files?

[Meir017]

I think it would be clearer to describe the request & response object using typescript
like this -

request: {
  method: 'GET' | 'POST' ...,
  url: string ...
}

[cstick]

@pke

Very cool. Could this be run from the command line? I guess not, since it depends on VSCode?
The tests are written directly in the http files?

Right, I do not believe there is a command line option but I do believe it is an open feature request. And yes, the tests are in the .http files, the sample above can be copy/paste into a .http file verbatim.

@Meir017
Sure, typescript looks more concise. I will look to add better descriptions but I want to avoid trying to capture the universe of all options; like all the HTTP status codes or all the HTTP methods, etc.

[pke]

So for CLI support we would have to extract the engine, that reads and executes http files?
This would also need to handle environments then somehow. So it does not seem like an easy task (without looking at the code base).

[pke]

Saw https://github.com/AnWeber/httpyac/ now, its a CLI for http files. Maybe we could port your test code to this too?

[AnWeber]

I might have a suggestion for improvement. You could connect this pull request with the pull request #674. Using NodeJS scripts it would be possible to use any test framework. The extension would only have to provide an interface for displaying the tests. For example it could look like this: Assert or chai.

[cstick]

I might have a suggestion for improvement. You could connect this pull request with the pull request #674. Using NodeJS scripts it would be possible to use any test framework. The extension would only have to provide an interface for displaying the tests. For example it could look like this: Assert or chai.

I would like to see some traction with this PR before considering something more complex. I like this plugin because it allows me to save .http files into source control alongside my APIs, something other HTTP clients do not have or have conveniently. But this plugin is missing something essential, the ability to run tests against API responses, for the many obvious reasons. I chose to import the Chai libraries because they are what Postman uses and therefore well known by many. And the pattern for writing tests should also be familiar for Postman users.

With consideration of adding more, different test libraries; if this testing capability is added, then I could see the ability to choose or add libraries via configuration of the plugin as a next step. I worry about the risk of allowing any module to be added, thinking about a bad actor and perhaps a future where .http files are part of automated functional testing.

[AnWeber]

I share your concerns with a malicious attacker, however your solution is not safe either.

GET https://cat-fact.herokuapp.com/facts/random HTTP/1.1

@tests
process.kill(2668);
const pathToFile = "";
console.info(process.mainModule.require('fs').readFileSync(pathToFile , 'utf8'))

For this reason, it should be ensured that code from unknown sources is executed only after trust has been granted.
I like the idea of using chai for automated tests. It would be very helpful in using the extension. And some traction with pr would be welcome.

[pke]

maybe @Huachao can take a look here after @cstick has rebased it?

[jupegarnica]

I love this feature , hopefully it will merged :)

[rngtng]

I only can recommend to checkout httpyac. It already has this feature which works great for me. Plus cli, plugins and more!

[cstick]

I only can recommend to checkout httpyac. It already has this feature which works great for me. Plus cli, plugins and more!

httpYac does look more promising; I will give it a shot this week, thx!

[herrberk]

Me and my team would love to see this get merged, we are considering adopting rest-client but the lack of test execution is a deal breaker. This would be a great addition to this plugin! @Huachao what are your thoughts?