Skip to content

feat: Support signed request to Elasticsearch service #151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 13, 2022
Merged

feat: Support signed request to Elasticsearch service #151

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 27 additions & 24 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,29 +54,32 @@ Serverless: GraphiQl: http://localhost:20002
Put options under `custom.appsync-simulator` in your `serverless.yml` file

| option | default | description |
| ------------------------ | -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| apiKey | `0123456789` | When using `API_KEY` as authentication type, the key to authenticate to the endpoint. |
| port | 20002 | AppSync operations port; if using multiple APIs, the value of this option will be used as a starting point, and each other API will have a port of lastPort + 10 (e.g. 20002, 20012, 20022, etc.) |
| wsPort | 20003 | AppSync subscriptions port; if using multiple APIs, the value of this option will be used as a starting point, and each other API will have a port of lastPort + 10 (e.g. 20003, 20013, 20023, etc.) |
| location | . (base directory) | Location of the lambda functions handlers. |
| refMap | {} | A mapping of [resource resolutions](#resource-cloudformation-functions-resolution) for the `Ref` function |
| getAttMap | {} | A mapping of [resource resolutions](#resource-cloudformation-functions-resolution) for the `GetAtt` function |
| importValueMap | {} | A mapping of [resource resolutions](#resource-cloudformation-functions-resolution) for the `ImportValue` function |
| functions | {} | A mapping of [external functions](#functions) for providing invoke url for external fucntions |
| dynamoDb.endpoint | http://localhost:8000 | Dynamodb endpoint. Specify it if you're not using serverless-dynamodb-local. Otherwise, port is taken from dynamodb-local conf |
| dynamoDb.region | localhost | Dynamodb region. Specify it if you're connecting to a remote Dynamodb intance. |
| dynamoDb.accessKeyId | DEFAULT_ACCESS_KEY | AWS Access Key ID to access DynamoDB |
| dynamoDb.secretAccessKey | DEFAULT_SECRET | AWS Secret Key to access DynamoDB |
| dynamoDb.sessionToken | DEFAULT_ACCESS_TOKEEN | AWS Session Token to access DynamoDB, only if you have temporary security credentials configured on AWS |
| dynamoDb.\* | | You can add every configuration accepted by [DynamoDB SDK](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB.html#constructor-property) |
| rds.dbName | | Name of the database |
| rds.dbHost | | Database host |
| rds.dbDialect | | Database dialect. Possible values (mysql/postgres) |
| rds.dbUsername | | Database username |
| rds.dbPassword | | Database password |
| rds.dbPort | | Database port |
| watch | - \*.graphql<br/> - \*.vtl | Array of glob patterns to watch for hot-reloading. |

| -------------------------- | -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| apiKey | `0123456789` | When using `API_KEY` as authentication type, the key to authenticate to the endpoint. |
| port | 20002 | AppSync operations port; if using multiple APIs, the value of this option will be used as a starting point, and each other API will have a port of lastPort + 10 (e.g. 20002, 20012, 20022, etc.) |
| wsPort | 20003 | AppSync subscriptions port; if using multiple APIs, the value of this option will be used as a starting point, and each other API will have a port of lastPort + 10 (e.g. 20003, 20013, 20023, etc.) |
| location | . (base directory) | Location of the lambda functions handlers. |
| refMap | {} | A mapping of [resource resolutions](#resource-cloudformation-functions-resolution) for the `Ref` function |
| getAttMap | {} | A mapping of [resource resolutions](#resource-cloudformation-functions-resolution) for the `GetAtt` function |
| importValueMap | {} | A mapping of [resource resolutions](#resource-cloudformation-functions-resolution) for the `ImportValue` function |
| functions | {} | A mapping of [external functions](#functions) for providing invoke url for external fucntions |
| dynamoDb.endpoint | http://localhost:8000 | Dynamodb endpoint. Specify it if you're not using serverless-dynamodb-local. Otherwise, port is taken from dynamodb-local conf |
| dynamoDb.region | localhost | Dynamodb region. Specify it if you're connecting to a remote Dynamodb intance. |
| dynamoDb.accessKeyId | DEFAULT_ACCESS_KEY | AWS Access Key ID to access DynamoDB |
| dynamoDb.secretAccessKey | DEFAULT_SECRET | AWS Secret Key to access DynamoDB |
| dynamoDb.sessionToken | DEFAULT_ACCESS_TOKEEN | AWS Session Token to access DynamoDB, only if you have temporary security credentials configured on AWS |
| dynamoDb.\* | | You can add every configuration accepted by [DynamoDB SDK](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB.html#constructor-property) |
| rds.dbName | | Name of the database |
| rds.dbHost | | Database host |
| rds.dbDialect | | Database dialect. Possible values (mysql/postgres) |
| rds.dbUsername | | Database username |
| rds.dbPassword | | Database password |
| rds.dbPort | | Database port |
| openSearch.useSignature | false | Enable signing requests to OpenSearch. The preference for credentials is config > environment variables > local credential file. |
| openSearch.region | | OpenSearch region. Specify it if you're connecting to a remote OpenSearch intance. |
| openSearch.accessKeyId | | AWS Access Key ID to access OpenSearch |
| openSearch.secretAccessKey | | AWS Secret Key to access OpenSearch |
| watch | - \*.graphql<br/> - \*.vtl | Array of glob patterns to watch for hot-reloading. |

Example:

Expand Down Expand Up @@ -257,7 +260,7 @@ This plugin supports resolvers implemented by `amplify-appsync-simulator`, as we

**Implemented by this plugin**

- AMAZON_ELASTIC_SEARCH
- AMAZON_ELASTICSEARCH
- HTTP
- RELATIONAL_DATABASE

Expand Down
73 changes: 73 additions & 0 deletions src/__tests__/data-loaders/ElasticDataLoader.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
import { PassThrough } from 'stream';
import * as AWS from 'aws-sdk';
import axios from 'axios';
import ElasticDataLoader from '../../data-loaders/ElasticDataLoader';

describe('data-loaders/ElasticDataLoader', () => {
beforeEach(() => {
jest.spyOn(AWS.HttpClient.prototype, 'handleRequest');
jest.spyOn(axios, 'request');
});

afterEach(() => {
AWS.HttpClient.prototype.handleRequest.mockClear();
axios.request.mockClear();
});

it('should send a request', async () => {
const loader = new ElasticDataLoader({
endpoint: 'https://my-elasticsearch-cluster.region.amazonaws.com',
});
axios.request.mockImplementation(async () => {
return { data: { hits: {} } };
});
const req = {
path: '[index]/_search',
operation: 'GET',
params: {
headers: {},
body: '{"query": { "match_all": {} }}',
},
};
const data = await loader.load(req);
expect(data).toEqual({ hits: {} });
});

it('should send a signed request', async () => {
const loader = new ElasticDataLoader({
endpoint: 'https://my-elasticsearch-cluster.region.amazonaws.com',
useSignature: true,
accessKeyId: 'fakeAccessKeyId',
secretAccessKey: 'fakeSecretAccessKey',
region: '',
});
const mockStream = new PassThrough();
let signedRequest;
AWS.HttpClient.prototype.handleRequest.mockImplementation(
(request, _options, callback) => {
signedRequest = request;
callback(mockStream);
},
);
const body = '{"query": { "match_all": {} }}';
const req = {
path: '[index]/_search',
operation: 'GET',
params: {
headers: {},
body,
},
};
process.nextTick(() => {
mockStream.emit('data', '{ "hits": {} }');
mockStream.end();
});
const data = await loader.load(req);
expect(signedRequest.headers.host).toEqual(
'my-elasticsearch-cluster.region.amazonaws.com',
);
expect(signedRequest.headers['Authorization']).toMatch(/^AWS4-HMAC-SHA256/);
expect(signedRequest.body).toEqual(body);
expect(data).toEqual({ hits: {} });
});
});
93 changes: 83 additions & 10 deletions src/data-loaders/ElasticDataLoader.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import axios from 'axios';
import * as AWS from 'aws-sdk';

export default class ElasticDataLoader {
constructor(config) {
Expand All @@ -7,20 +8,92 @@ export default class ElasticDataLoader {

async load(req) {
try {
const { data } = await axios.request({
baseURL: this.config.endpoint,
url: req.path,
headers: req.params.headers,
params: req.params.queryString,
method: req.operation.toLowerCase(),
data: req.params.body,
});

return data;
if (this.config.useSignature) {
const signedRequest = await this.createSignedRequest(req);
const client = new AWS.HttpClient();
const data = await new Promise((resolve, reject) => {
client.handleRequest(
signedRequest,
null,
(response) => {
let responseBody = '';
response.on('data', (chunk) => {
responseBody += chunk;
});
response.on('end', () => {
resolve(responseBody);
});
},
(err) => {
reject(err);
},
);
});
return JSON.parse(data);
} else {
const { data } = await axios.request({
baseURL: this.config.endpoint,
url: req.path,
headers: req.params.headers,
params: req.params.queryString,
method: req.operation.toLowerCase(),
data: req.params.body,
});

return data;
}
} catch (err) {
console.log(err);
}

return null;
}

async createSignedRequest(req) {
const domain = this.config.endpoint.replace('https://', '');
const headers = {
...req.params.headers,
host: domain,
'Content-Type': 'application/json',
'Content-Length': Buffer.byteLength(req.params.body),
};
const endpoint = new AWS.Endpoint(domain);
const httpRequest = new AWS.HttpRequest(endpoint, this.config.region);
httpRequest.headers = headers;
httpRequest.body = req.params.body;
httpRequest.method = req.operation;
httpRequest.path = req.path;

const credentials = await this.getCredentials();
const signer = new AWS.Signers.V4(httpRequest, 'es');
signer.addAuthorization(credentials, new Date());

return httpRequest;
}
Copy link
Member

@bboure bboure Feb 15, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also add the option to load the keys from env vars and/or local credentials file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.
I added elasiticsearch.useSignature option and modified to load credentials from config, env vars or local file.

By the way, I'm using the name elasticsearch in the config, would openSearch be more appropriate?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! thanks!

Yes openSearch is probably what we should start calling things now ✅

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
I changed the name from elasticsearch to openSearch.


async getCredentials() {
const chain = new AWS.CredentialProviderChain([
() => new AWS.EnvironmentCredentials('AWS'),
() => new AWS.EnvironmentCredentials('AMAZON'),
() => new AWS.SharedIniFileCredentials(),
]);
if (this.config.accessKeyId && this.config.secretAccessKey) {
chain.providers.unshift(
() =>
new AWS.Credentials(
this.config.accessKeyId,
this.config.secretAccessKey,
),
);
}
return new Promise((resolve, reject) =>
chain.resolve((err, creds) => {
if (err) {
reject(err);
} else {
resolve(creds);
}
}),
);
}
}
5 changes: 5 additions & 0 deletions src/getAppSyncConfig.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,11 @@ export default function getAppSyncConfig(context, appSyncConfig) {
};
}
case SourceType.AMAZON_ELASTICSEARCH:
return {
...context.options.openSearch,
...dataSource,
endpoint: source.config.endpoint,
};
case SourceType.HTTP: {
return {
...dataSource,
Expand Down
1 change: 1 addition & 0 deletions src/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,7 @@ class ServerlessAppSyncSimulator {
accessKeyId: 'DEFAULT_ACCESS_KEY',
secretAccessKey: 'DEFAULT_SECRET',
},
openSearch: {},
},
get(this.serverless.service, 'custom.appsync-simulator', {}),
);
Expand Down