support TypeScript for backend functions

[troygoode]

Is your feature request related to a problem? Please describe.

I use the TypeScript generation features for my front-end client, but am frustrated that any of the Lambda functions I create via amplify add api or amplify add function must be written in vanilla JavaScript. This also makes it very difficult to share code between the backend and frontend (note that many of the auto-generated files created by amplify cli use syntax like export default blah that is invalid for use in the backend files).

Describe the solution you'd like

Switch to using tsc to compile the files found in amplify/backend/function/*/src/ so that we can opt-in to type-safety by changing our file extension from .js to .ts or .tsx, similar to how Create React App has handled this in version 2.0.0+.

Describe alternatives you've considered

Alternatively, add (or document) how we can hook into the build process for amplify/backend/function/* so we can inject a TypeScript transpilation step ourselves.

[awjreynolds]

Really want this feature. I know I'm not adding much to the conversation but there needs to be options within the cli to scaffold typescript outputs for all aspects of an amplify solution.

[ajhool]

Have you considered using the "postinstall" hook in npm? I have not tried using it but am about to give it a shot.

Amplify uses a "amplify function build" command and the docs say it calls npm install under the hood. The npm docs say that npm run postinstall is called automatically after npm install, so it looks promising. This might require installing typescript + build tools on the build machine without utilizing the function's package.json, but that's not too bad.

[ajhool]

EDIT: Having used this solution for a while... it's a little fragile. However, I think it is a promising concept and is a good place to start.

I was able to create a simple typescript lambda function that imports and configures Amplify and invoke it locally using the amplify function invoke myFunction command. I have not tested it in the backend, yet, and might not be able to for a few days.

This might look like a long procedure, but it should take about 2 minutes and once it is configured, it should be easy to maintain. While it isn't a native amplify plugin, it does feel like an npm-compliant solution, so I think it's a legitimate, stable way to do this.

WARNING: THIS WILL OVERWRITE CODE IN YOUR FUNCTION'S "src" directory (eg. amplify/backend/function/myFunction/src)

Strategy

Inside the lambda function scaffold (eg. amplify/backend/function/myFunction), create a typescript src directory (tsc-src) that transpiles into the src directory used by Amplify. Transpilation is triggered by the postinstall hook in myFunction/package.json; thus, transpilation will be triggered by Amplify at all the appropriate times.

Procedure

1. Create a sister directory to amplify/backend/function/myFunction/src called function/myFunction/tsc-src
2. Create a file at function/myFunction/src/postinstall.sh:

#!/bin/bash

cd ../tsc-src
tsc
yarn install

3. Make postinstall.sh executable (on my mac the command is): chmod u+x postinstall.sh

4. Update function/myFunction/src/package.json with the postinstall script:

{
  "name": "myFunction",
  "version": "2.0.0",
  "description": "Lambda function generated by Amplify",
  "main": "index.js",
  "license": "UNLICENSED",
  "scripts": {
    "postinstall": "./postinstall.sh"
  }
}

5. In function/myFunction/tsc-src (eg. cd function/myFunction/tsc-src) run the command tsc --init

6. Edit function/myFunction/tsc-src/tsconfig.json to include:

    "outDir": "../src",                        /* Redirect output structure to the directory. */

7. Write your typescript in the tsc-src folder. If you have javascript in your src folder, you should copy it to a different folder because this setup might overwrite that src folder when you build the lambda function. I would recommend starting with a very simple typescript file so you know the build process is configured properly

8. Install build dependencies You will need to do this on every build machine!

yarn global add typescript

WARNING: THIS NEXT STEP WILL OVERWRITE CODE IN YOUR FUNCTION'S "src" directory (eg. amplify/backend/function/myFunction/src)

9. To test everything (I had to use yarn because npm is bugging out on my machine):

// amplify function build myFunction
cd amplify/backend/function/myFunction/src
yarn install
amplify function invoke myFunction
... You should see javascript files being generated in amplify/backend/function/myFunction/src ...

You might run into a typescript error issue with aws-exports.js, I solved that by adding this line to postinstall.sh before tsc. The relative path should actually be the same for your project.:
cp ../../../../../src/aws-exports.js ./aws-exports.ts

[prabu-ramaraj]

I noticed that you have to update package.json everytime before calling amplify function build for the typescript to compile into JS. I know we are working around this problem and it would be nice if Amplify CLI automatically compiles Typescript to JavaScript out of the box.

[andfk]

Or if at least they provide us with a hook to run a npm script or similar... :)

[royalaid]

FWIW I am building an amplify app with Clojurescript and would love the option to easily write my node.js functions in the same language as the rest of my stack. If support for typescript is considered I hope with that would come in the form that supports all compile to js languages.

[mrcoles]

This would be awesome to have. Lambda functions are hard to test locally (relevant: issue with amplify function invoke) and Typescript would at least create some more confidence that simple errors aren’t going to make it into the code.

Also, natural next steps (I have an existing issue about sharing code between lambda functions), and, with Typescript in the lambda functions, there should also be an easy way to share types between server and client-side code (isomorphic types?), mainly for API calls. Maybe that’s as simple as them living in the lambda function and a lengthy import from the client code, e.g., import { MyType } from '../amplify/backend/functions/myfunction/src/app';, but making sure that can be done would be really helpful.

[kaustavghosh06]

@mrcoles You can actually use Typescript with the new Build options feature. Please take a look out here - https://aws-amplify.github.io/docs/cli-toolchain/usage#build-options

[mrcoles]

@kaustavghosh06 cool, thanks for sharing this update! Does that mean this issue should be updated or are some more things in the works on this? Also, does this require a minimum aws-amplify installed in the project root’s package.json or something like that?

Some things from reading that example:

1. is the choice of lib for the es6 code and for it to compile into src just a stylistic choice dependent on the specific babel invocation in "amplify:generateReport"? I like the idea of working in "src" and having the code build into "dist", but would I run into issues doing that? (Also it has always felt weird that package.json is inside "src/")
2. I didn’t immediately correctly parse the words "project root" for the package.json updates with babel devDependencies and scripts.
3. Are any scripts that start with "amplify:" run during build (basically the npm-run-all syntax)? (Just trying to remove any magic and understand what choices the build is making)

[pechisworks]

Is it possible to test this locally?
Referring to the documentation you have to use amplify push.

[RossWilliams]

@kaustavghosh06 I had the exact same 3 problems as @mrcoles when reading the documentation.

[kaustavghosh06]

You can read about adding lambda functions in Typescript as a part of this blog - https://servicefull.cloud/blog/amplify-ts/

[dtelaroli]

+1

[ngnathan]

Has anyone tried TypeScript recently in Amplify Lambda functions with Promises? I'm receiving errors that I can't get past.

If I try to use Promise directly (e.g. new Promise()), I get the following error:

index.ts:6:38 - error TS2585: 'Promise' only refers to a type, but is being used as a value here. Do you need to change your target library? Try changing the `lib` compiler option to es2015 or later.

6     const wait = (ms: number) => new Promise(res => setTimeout(res, ms));
                                       ~~~~~~~

If I try to use async/await:

error TS2468: Cannot find global value 'Promise'.
index.ts:91:35 - error TS2705: An async function or method in ES5/ES3 requires the 'Promise' constructor.  Make sure you have a declaration for the 'Promise' constructor or include 'ES2015' in your `--lib` option.

91     const updateDocument = async () => {

Even in the blog https://servicefull.cloud/blog/amplify-ts/, if I use that example (with their tsconfig.json) with the following code:

import { APIGatewayProxyHandler } from 'aws-lambda';
import 'source-map-support/register';

export const handler: APIGatewayProxyHandler = async (event, _context) => {
    console.log(event);
    const wait = (ms: number) => new Promise(res => setTimeout(res, ms));
    console.log('data', wait);
    return {
        statusCode: 200,
        body: JSON.stringify({
            message: 'Amplify function in Typescript!',
            input: event
        })
    };
};

It throws the same error as the first one above. I've tried this with numerous tsconfig.json configurations and none of them seem to make a difference.

[ngnathan]

Nevermind on the post above. Turns out that in the blog post https://servicefull.cloud/blog/amplify-ts/, the tsc ./*.ts command in package.json skips the tsconfig.json that you created and uses the default settings, so I switched it to just 'tsc' and it works fine (or you can use tsc --project tsconfig.json).

[ChrisSargent]

@kaustavghosh06 - is it possible to run this same behaviour when running amplify mock too? I have setup some build scripts that run webpack on my files so that I can now have Typescript and use shared libraries etc. However, the scripts are only run with amplify push. In the time being I will create my own little scripts to handle this but would be good if it was baked in.

[adilusman51]

@kaustavghosh06 - is it possible to run this same behaviour when running amplify mock too? I have setup some build scripts that run webpack on my files so that I can now have Typescript and use shared libraries etc. However, the scripts are only run with amplify push. In the time being I will create my own little scripts to handle this but would be good if it was baked in.

@ChrisSargent +1

amplify mock function does not execute Build Options for functions properly

[ChrisSargent]

FWIW, I used package script to achieve this for now, something along the lines of: "mock:api": "concurrently \"amplify mock api\" \"tsc --watch\"". I also actually treat each function as its own yarn workspace and put my real source code in a function/functionname/lib folder. This then compiles with webpack in to the src folder which is where amplify needs it to be. The code in the src folder is completely bundled and can be optimised / tree shaken etc. with webpack. So I have a folder structure like this:

function
- functionname
-- lib
-- index.ts // exports my handler
-- src
-- index.js // minified & bundled
-- package.json // empty
- package.json // has the dependencies for index.ts in /lib

Then, because of Yarn workspaces, I can also import local shared packages - which, of course, get bundled in to the final index.js code. And, since the 'empty' package.json file un the src folder doesn't set any dependencies, none of my local packages are missing when pushing to the cloud.

[danielblignaut]

My 2 cents on running amplify with Typescript... (be prepared for some code copy and pasting)

1. on the entire application, make use of yarn workspaces... I configure my package.json workspaces like this

"workspaces": {
    "packages": [
      "amplify/backend/function/**/src"
    ],
    "nohoist": [
      "**"
    ]
  },

3. add a root tsconfig.json like this

{
  "compilerOptions": {
    "sourceMap": true,
    "target": "es2017",
    "moduleResolution": "node",
    "esModuleInterop": true,
    "types": ["node"],
	"module": "commonjs",
	"paths": {
		
	},
	"baseUrl": "."
  },
  "exclude": [
	  "node_modules",
	  "**/build",
	  "**/dist"
	],
}

4. Everytime you add a lambda function, edit the {name}-cloudformation-template.json and set the Handler property not to reference your ts file but your final built js file. (build/index.handler instead of just index.handler)

"Handler": "build/index.handler", 

inside the src folder for the lambda, edit the lambda package.json

{
  "name": "uniqueName",
  "version": "2.0.0",
  "description": "Lambda function generated by Amplify",
  "main": "index.js",
  "license": "Apache-2.0",
  "dependencies": {
    "source-map-support": "^0.5.16"
  },
  "devDependencies": {
    "@types/aws-lambda": "^8.10.36",
    "@types/node": "^12.12.14",
    "typescript": "^3.7.2"
  }
}

inside the src folder for the lambda, add a tsconfig file that extends your root tsconfig:

{
  "extends": "../../../../../tsconfig.json",
  "compilerOptions": {
    "outDir": "./build",
    "rootDir": "./"
  }
}

update your root package.json for the entire amplify project to compile your new lambda on push

"scripts": {
	"amplify:lambdaName": "cd amplify/backend/function/lambdaName/src && npx tsc"
  },

5. Everytime you add a lambda layer:

inside the lib/nodejs file, edit the package.json to be like so

{
  "name": "uniqueName",
  "version": "2.0.0",
  "description": "Lambda function generated by Amplify",
  "main": "index.js",
  "license": "Apache-2.0",
  "dependencies": {
    "source-map-support": "^0.5.16"
  },
  "devDependencies": {
    "@types/aws-lambda": "^8.10.36",
    "@types/node": "^12.12.14",
    "typescript": "^3.7.2"
  }
}

inside the lib/nodejs file, add a tsconfig.json file

{
	"extends": "../../../../../../tsconfig.json",
	"compilerOptions": {
	  "outDir": "./build",
	  "rootDir": "./",
	  "declaration": true
	}
}
  

** note the additional declaration output **

edit your root package.json to build the lambda layer on amplify push by add this under the scripts section

"amplify:layerName": "cd amplify/backend/function/layerName/lib/nodejs && npx tsc"

edit your root tsconfig.json to add a reference to you declaration file from the lambda layer

"paths": {
		"/opt/nodejs/build/exampleOutputJsFileFromInLambdaLayerBuildFile": ["./amplify/backend/function/layerCommonDynamoDb/lib/nodejs/build/declarationFileNameFromLambda.d.ts"]
	},

** note: exampleOutputJsFileFromInLambdaLayerBuildFile is the final build file from your lambda layer... fileName is important as it has to match aws lambdas /opt/nodejs folder structure to work on AWS correctly otherwise your directory references will be wrong **

you can now use your lambda layer in your lmbda functions:

import commonLib from '/opt/nodejs/build/exampleOutputJsFileFromInLambdaLayerBuildFile'

I haven't tried using this with amplify mock, i think a lot of work needs to be done on the local amplify mocking / testing environment anyway so rather use a multi environment AWS workflow... add the sourcemap import to the top of your lambda as well for better debugging experience.

[grant-d]

@danielblignaut thank you so much. I had this all working in a very hacky manner. But this is clean and simple, appreciate the advice.

[r0zar]

I'm actually against adding this functionality until Typescript is supported natively by AWS Lambda- hear me out...

The ideal scenario would be that typescript would be pushed up as is.

We'd then get type support directly inside the Lambda editor on the AWS Console.

It would also really suck to have Amplify spend time on this to then see Typescript support pop up in AWS Lambda.

[hisham]

@r0zar has a point.

I personally develop all my backend code as an independent typescript node package. You can then use all the things you like such as jest for unit testing. The lambda then installs that private node package and becomes a very thin javascript layer on top of that node package built in typescript.

This has worked out well and lets us develop and test things independently of the amplify workflow. The package also becomes very re-usable for scenarios outside of amplify (such as our own cli to interact with our backend).

[danielblignaut]

@r0zar I think it's a very low / non-existant priority for the lambda team as there is no "typescript" runtime in reality and even if so, it would probably be less performant if you want to run something like ts-node v building your package and running it directly as building with webpack gives you tree shaking, and other features to increase speed and reduce package size. You can also get runtime type support (for stack traces) with source maps already.

personally, as a typescript developer, I've moved away from Amplify and have instead adopted AWS CDK which is another infrastructure as code library written by the AWS team. This library is based in typescript (although there are other language variations) and all the latest features come to typescript first. I also use lerna for a mono-repo structure to hold all my lambda's and build their typescript packages independently. With CDK you can choose the file location of your lambda code so I just add my lambda's as dependency's to the cdk package and use require.resolve to find their built code in the node_modules and deploy. Honestly works like a dream, does not feel hacky at all and personally my experience with CDK as a typescript dev has been great especially on bigger projects.

[askaribragimov]

@danielblignaut, I am curious about your implementation, do you have an open source example?

[ziggy6792]

My 2 cents on running amplify with Typescript... (be prepared for some code copy and pasting)

1. on the entire application, make use of yarn workspaces... I configure my package.json workspaces like this

"workspaces": {
    "packages": [
      "amplify/backend/function/**/src"
    ],
    "nohoist": [
      "**"
    ]
  },

1. add a root tsconfig.json like this

{
  "compilerOptions": {
    "sourceMap": true,
    "target": "es2017",
    "moduleResolution": "node",
    "esModuleInterop": true,
    "types": ["node"],
	"module": "commonjs",
	"paths": {
		
	},
	"baseUrl": "."
  },
  "exclude": [
	  "node_modules",
	  "**/build",
	  "**/dist"
	],
}

1. Everytime you add a lambda function, edit the {name}-cloudformation-template.json and set the Handler property not to reference your ts file but your final built js file. (build/index.handler instead of just index.handler)

"Handler": "build/index.handler", 

inside the src folder for the lambda, edit the lambda package.json

{
  "name": "uniqueName",
  "version": "2.0.0",
  "description": "Lambda function generated by Amplify",
  "main": "index.js",
  "license": "Apache-2.0",
  "dependencies": {
    "source-map-support": "^0.5.16"
  },
  "devDependencies": {
    "@types/aws-lambda": "^8.10.36",
    "@types/node": "^12.12.14",
    "typescript": "^3.7.2"
  }
}

inside the src folder for the lambda, add a tsconfig file that extends your root tsconfig:

{
  "extends": "../../../../../tsconfig.json",
  "compilerOptions": {
    "outDir": "./build",
    "rootDir": "./"
  }
}

update your root package.json for the entire amplify project to compile your new lambda on push

"scripts": {
	"amplify:lambdaName": "cd amplify/backend/function/lambdaName/src && npx tsc"
  },

1. Everytime you add a lambda layer:

inside the lib/nodejs file, edit the package.json to be like so

{
  "name": "uniqueName",
  "version": "2.0.0",
  "description": "Lambda function generated by Amplify",
  "main": "index.js",
  "license": "Apache-2.0",
  "dependencies": {
    "source-map-support": "^0.5.16"
  },
  "devDependencies": {
    "@types/aws-lambda": "^8.10.36",
    "@types/node": "^12.12.14",
    "typescript": "^3.7.2"
  }
}

inside the lib/nodejs file, add a tsconfig.json file

{
	"extends": "../../../../../../tsconfig.json",
	"compilerOptions": {
	  "outDir": "./build",
	  "rootDir": "./",
	  "declaration": true
	}
}
  

** note the additional declaration output **

edit your root package.json to build the lambda layer on amplify push by add this under the scripts section

"amplify:layerName": "cd amplify/backend/function/layerName/lib/nodejs && npx tsc"

edit your root tsconfig.json to add a reference to you declaration file from the lambda layer

"paths": {
		"/opt/nodejs/build/exampleOutputJsFileFromInLambdaLayerBuildFile": ["./amplify/backend/function/layerCommonDynamoDb/lib/nodejs/build/declarationFileNameFromLambda.d.ts"]
	},

** note: exampleOutputJsFileFromInLambdaLayerBuildFile is the final build file from your lambda layer... fileName is important as it has to match aws lambdas /opt/nodejs folder structure to work on AWS correctly otherwise your directory references will be wrong **

you can now use your lambda layer in your lmbda functions:

import commonLib from '/opt/nodejs/build/exampleOutputJsFileFromInLambdaLayerBuildFile'

I haven't tried using this with amplify mock, i think a lot of work needs to be done on the local amplify mocking / testing environment anyway so rather use a multi environment AWS workflow... add the sourcemap import to the top of your lambda as well for better debugging experience.

@danielblignaut thanks so much for your reply. I found it incredibly helpful. I am finally able to use my typescript lambda layer properly.

Could you please elaborate on your final comment about not using this with amplify mock. (It doesn't work with amplify mock by the way, as soon as I import my lambda layer library and run amplify mock it just hangs and then times out).

So I am just wondering how you test your code before pushing to amplify. Do you still test it locally? Is there a way to test this locally? (A better way than amplify mock???) What exactly do you mean by multi environment AWS workflow?

Thanks again :)

[danielblignaut]

Hey @ziggy6792 ,

I can’t quite remember the reasoning behind the last comment. But to provide some further points:

1. I actually ended up avoiding layers all together. I found amplify struggled to maintain correct layer versioning with the remote AWS account leading to me deploying lambda functions referencing out of date lambda layers, etc. I opened up a GitHub issue on Amplify code base so should be able to find if it’s been resolved or not (this was when layers first launched).
2. I’ve moved away from using layers in JavaScript. My reasoning: I feel that layers are great to provide common library access for most languages! However, JavaScript has awesome tooling from its community and I feel that for that reason, layers is least advantageous to JavaScript language v others because of yarn, lerna, etc. I think a better JavaScript and especially typescript approach is to use lerna in a monorepo structure, create the lambda layer as a normal package in that monorepo and require it in your package.
3. As a typescript developer, I’ve moved away from amplify and now use CDK. I recommend for typescript devs in particular you check this library out and especially for bigger projects. You have to write a little more code for your infrastructure but you gain more flexibility in project design and you win first class typescript support.
4. Testing is always hard to emulate. It’s really important that you test on an AWS (non production stage). No tool I’ve used: localstack, AWS local step functions, AWS dynamodb actually recreates the real AWS service without some limitation or bug and there’s a MASSIVE gap in development experience here. Personally, I test all my graphql APIs by writing queries that test the appsync server directly. Last time I checked, and that’s a while ago, amplify actually just runs a local server that proxies the remote appsync url... it’s hard to test appsync locally as it runs a custom Apache VTL engine which is hard to recreate. AWS team : appsync local server would be awesome FYI. Besides this you can still write unit tests for all your lambda code and finally for integration testing on lambda, I run an express server with some custom middleware to make inbound requests / events look like api gateway events that lambda receives and also wrap the callback function. I can share this to assist if that helps but let me know as will take some time (will have to make a reproduction of a closed source project) generally for local testing I use the following tooling:

• AWS dynamodb local
• My own express “lambda” server
• AWS step functions local (set it to connect to dynamodb and my own express lambda server)
• Local stack for cognito services and S3... warning that lots of localstack features I found don’t work.
• At some point I'd like to work on a local appsync implementation for dynamodb and lambda VTL support... If you look at amplify github project, there's a package called "graphql-mapping-template" which is a clever start at wrapping Apache VTL language into re-usable javascript code... I plan to look at how we can grow that library further and either provide more development features when working in VSCODE or instead of boiling it down to APACHE VTL, transpile it to javascript and run a graphql server that understands it and simulates appsync locally. But for now, as said I write graphql query and mutations and my tests run on an AWS sandbox environment... also, I use snapshot tests here to see if my vtl templates ever change on deploy.

EDIT: Here's the lambda layers issue which seems resolved now: #4963

[ziggy6792]

Hey @danielblignaut

Thanks so much for your very detailed reply and especially for pointing me towards CDK.

I am still trying to setup a backend where I have several lambda functions which share a common lib (which will ultimately talk to dynamo-db).

I got pretty far with CDK and tried to follow you instructions as best as I could.
I created a very simple demo with one hello-world lambda importing from one common my-lib package.
https://github.com/ziggy6792/aws-cdk-lambda-shared-package

If I deploy this stack to AWS and run my hello-world lambda (by testing from the AWS console) it works! (It successfully imports my-lib does not error).

However once again I have the problem of mocking locally.

I have tried to use this method (which I found here) to mock locally (this method works fine when I don't import my common my-lib).

• cdk synth --no-staging > template.yml (to find the logical lambda function id = HelloWorldLambda5A02458E)
• sam local invoke HelloWorldLambda5A02458E

But I get an error

{"errorType":"Runtime.ImportModuleError","errorMessage":"Error: Cannot find module 'my-lib/MyLib'\nRequire stack:\n- /var/task/index.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js"}

My questions are...

1. Am I on the right track? Is the setup I have close to what you were suggesting? Any improvements I should make?
2. Do you know how I can test my lambda function locally (not being able to test locally is a deal breaker for me)?

Thanks a lot 👍

EDIT: I found this which is an example (using CDK) of how to setup shared code in a lambda layer (includes deploying to stack and testing locally). What do you think @danielblignaut? It seems pretty complicated to me but maybe I can get my head round it.
I would still really like to see an example of your suggested approach "... use lerna in a monorepo structure, create the lambda layer as a normal package in that monorepo and require it in your package." as long as it can work with running locally too.

[danielblignaut]

@ziggy6792 @askaribragimov ,

Some feedback:

• At the moment my CDK project is closed source, but I will work to get a minimum reproduction of the lambda proxy, how I test my lambdas using jest, and deploying multi environment CDK projects
• For a great introduction to AWS codepipeline with CDK and deploying to multi-env aws accounts see this tutorial https://docs.aws.amazon.com/cdk/latest/guide/cdk_pipeline.html
• Mocking a lambda for local development involves re-creating a lightweight web server that can execute the lambda. For now, take a look at https://github.com/localstack/localstack and particularly the lambda documentation (localStack can be buggy though so here's another project to help with lambda mocking that I found from a quick google https://www.npmjs.com/package/lambda-local
• a brief tutorial on mocking api gateway lambdas https://steveholgado.com/aws-lambda-local-development/
• More important than true lambda mocking for me is having a strong testing environment for automated unit, integration and end to end tests

Here's an example CDK repo I've setup with 2 lambdas, a common library and a CDK project. Includes all the bells and whistles except multi-aws account deployment (point 2) and testing environment could use a lot of work.

https://github.com/danielblignaut/cdk-monorepo-example

[ziggy6792]

@danielblignaut wow this is incredible.
Thank you so much!!!

[jcbdev]

I found the best way to test the lambda functions locally is to bypass trying to run them as lambdas completely... I know sounds crazy but stick with me!

The key is really adding this to your jest.config.js to redirect the module imports on the js files in your backend folder

moduleNameMapper: {
    '^.+\\.(css|styl|less|sass|scss|png|jpg|ttf|woff|woff2)$':
      'jest-transform-stub',
    '^/opt/(.*)$': '<rootDir>/amplify/backend/function/core/opt/$1', <--This line
  },

I then have a set up function called configEnv() that i call in beforeAll() setup function

import AWS from 'aws-sdk';
import * as config from '../../../aws-exports';
import * as meta from '../../../amplify/backend/amplify-meta.json';
import * as localEnv from '../../../amplify/.config/local-env-info.json';

export const configEnv = () => {
  var credentials = new AWS.SharedIniFileCredentials({profile: 'amplify'});
  AWS.config.credentials = credentials;
  AWS.config.region = config.default.aws_appsync_region;
  process.env.API_MYAPP_GRAPHQLAPIIDOUTPUT = meta.api.myapp.output.GraphQLAPIIdOutput;
  process.env.API_MYAPP_GRAPHQLAPIENDPOINTOUTPUT = config.default.aws_appsync_graphqlEndpoint;
  process.env.ANALYTICS_ROARNOTIFICATIONS_ID = config.default.aws_mobile_analytics_app_id;
  process.env.ANALYTICS_ROARNOTIFICATIONS_REGION = config.default.aws_mobile_analytics_app_region;
  process.env.REGION = config.default.aws_appsync_region;
  process.env.ENV = localEnv.envName;
}

and then I just set up a test:

import {handler} from '../../../amplify/backend/function/someHandler/src/index';

describe("Test some handlers", () => {
  beforeAll(async () => {
    configEnv();
  });

  it("Should run some handler", async () => {
    const event = {
      typeName: 'Mutation',
      fieldName: 'someHandler',
      arguments: {
        input: {
          someData: 12335
        }
      },
      identity: {
        username: '<cognitoId>'
      }
    }

    const response = await handler(event);

    expect(response).toBeTruthy();
  })
})

Obviously this will run as integration tests hitting your actual servers so if you need to hit a local stack then change the configEnv(). Personally I think real integration tests are highly underrated but I will write a bunch of unit and component tests with the various parts mocked out and then leave one or two full integration tests in to run against a proper stack. This is great in amplify because you can use a sandbox environment for most stuff. I don't really like setting up a local stack to "emulate" because they rarely behave like a real environment and I find most time lost on projects is debugging issues with the testing environment rather than the actual code. Since taking this approach and dropping localstack/amplify mock etc. I have saved tons of time in debugging.

This jest module redirect technique also works with layers etc. which the current amplify mock and such don't so this is much much easier. Also works well with typescript because you don't need to build the functions before you test locally if your jest is setup to support typescript. You only need your amplify build command to run tsc in packages.json in the root project - "amplify:someHandler": "cd amplify/backend/function/someHandler/src && tsc

[ziggy6792]

@danielblignaut

Some feedback on your project. Firstly thanks so much again. This is by far the best way I have seen to locally mock and test lambdas. My productivity is going to increase so much with this setup.

• CDK Pipeline. I did the tutorial.
  Now I want to add a pipeline to this project. Would you suggest to put the frontend and backend in the same git repo and the same pipeline?
  So the pipeline would be something like;

Commit and push changes
          |
Create dev stage (deploy front end + lambdas + gateway)
          |
Manual approval
          |
Create prod stage (deploy front end  + lambdas + gateway)

I guess the disadvantage of this is every-time I commit a change to my front my whole backend will be redeployed too (and vice versa).
What do you think?

• Absolute imports. It seems I can't use absolute imports in my lambdas right? E.g: below image I am trying to import ./example/example using the absolute path (from lambda-a src) and it doesn't work (I also tried to import absolutely from the root packages/lambda-a/src/example/example and that didn't work either). This isn't a huge problem, I just need to remember not to use them.

I tried adding

"compilerOptions": {
     ...
    "baseUrl": "./",
  },

To lambda-a tsconfig.build.json but it didn't help.

[danielblignaut]

@ziggy6792 ,

no problem. Glad it's helping! I've built other tooling around appsync and local environment setup that I'm hoping to publicly share but there's a lot of work documentation that I need to create first. Glad to hear it's helping someone! Your comments:

• I'd generally seperate backend and frontend into 2 seperate projects or git repos. Whilst using lerna and monorepo's you can do this, I'd generally advise against it. The main reason for lerna is to be able to improve development efficiency when working on multiple packages that are linked together without having to work on one package, build, deploy it and then switch to another npm package project to work on. Whilst you may have some shared code for your GraphQL schema or something similar, I fear it would run the risk of promoting bad habits by sharing code from backend to front end that generally shouldn't happen. Also your environments are very different, one is browser based and other is server nodeJS. But again, this is personal preference my side. If you would like to be able to re-use the same build pipeline across projects or in general, if you would like to re-use your CDK code, I'd advise looking at Constructs in CDK. These are how they promote creating re-usable higher level components not tied to a specific stack and scope. It may seem natural to just re-use stacks but I've read in a few github issues that constructs are the way to go. Finally, I personally use nextJS and vercel for all frontend REACT.JS projects as the setup is so quick and simple so my experience with deploying frontends on AWS is not that in-depth.
• I've never used absolute imports TBH. The only import tooling I use that's not in that project is usage of typescript paths in my test environment so that I don't have "../../../../" to get to the root of the project where I keep some test utils. This can be done by configuring jest and the tsconfig.test.json . This might not be the answer you're looking for but a paths setup may help for what you're looking for.

When I have a chance I'll add my CDK pipeline tools to that project that may help as well.

[ziggy6792]

Awesome thanks 👍

[ziggy6792]

@danielblignaut

I have done quite a lot of work in the foundations for my app running using cdk (setup some lambdas, api gateway, cognito user pool etc.. ) and I have a react frontend which can authenticate with cognito and then call my api.

I have a frontend repo and a backend repo as you suggested and I would now like to setup a pipeline for both; so that changes merged to master are automatically deployed to dev and then prod (after a manual approval step).

It's pretty clear to me how to setup the backend pipeline as it is just an extension of the starter project you already shared. However I am less clear about how to setup the frontend pipeline.

Would you suggest a folder structure like this?

So I would have 2 yarn workspaces

1. The CDK app that describes the pipeline (deploying my frontend to S3 for dev and prod stages)
2. The actual frontend

For me the complicated bit is going to be configuring the tsconfig and .eslintrc correctly (as one workspace will have to be setup for react and running in the browser and the other workspace will be a normal node js setup).

I can probably get this working on my own. Just wanted to know if I am on the right tracks here. Or is there a better/simpler solution? Also if you have an example of typescript react frontend that is deployed to S3 using a CDK pipeline that would be amazing.

Thanks so much for your help so far.

[GeorgeBellTMH]

This seems like a fairly straight forward feature to add...isn't it just a different template in addition to the ones for nodeJS and python etc...someone just needs to provide the simplest typescript project that works on lambda and add it to the menu I would think?

[acusti]

after seeing many of the approaches here (thanks to all for documenting them!), i’ve landed on a setup using esbuild --bundle and yarn workspaces that minimizes the size of the lambda functions that get deployed. my lambdas all have empty node_modules thanks to yarn workspace dependency hoisting and minimal script sizes thanks to the bundling and tree-shaking via esbuild (total size of my lambdas range between 20KB–300KB including aws-sdk v3 dependencies). here’s the full write-up: https://www.acusti.ca/blog/2022/03/30/writing-your-amplify-functions-in-typescript-via-esbuild-and-yarn-workspaces/

the most important bits are the build script in each amplify function’s package.json, which runs tsc for type-checking then esbuild for transpiling / bundling (see the blog post for details on the esbuild options):

  "scripts": {
    "build": "tsc -noEmit && esbuild *.ts --main-fields=module,main --bundle --platform=node --external:@aws-sdk/signature-v4-crt --outdir=./"
  }

and the per-function build script in the amplify root package.json:

  "scripts": {
    "amplify:<functionName>": "cd amplify/backend/function/<functionName>/src && yarn && yarn build"
  }

i also use baseUrl in my amplify root tsconfig and extend it from the function-specific tsconfigs so that i can easily and cleanly import GraphQL queries and mutations and types (see the blog post for full tsconfig.json contents):

import { GetItemQuery, GetItemQueryVariables } from 'API.js';
import { getItem } from 'graphql/queries.js';

[chrisl777]

We typically use Webpack to bundle up the solution. Sharing our settings here as well:

Root package.json:

"scripts": {
    "amplify:<functionName>": "cd amplify/backend/function/<functionName> && yarn install && webpack && cd -"
  },

The function's tsconfig.json:

{
  "compilerOptions": {
    "lib": [
      "es2017",
      "DOM"
    ], 
    "target": "es2017", 
    "module": "commonjs",                     
    "moduleResolution": "node",            
    "baseUrl": "./",
    "sourceMap": true,
    "esModuleInterop": true,
    "resolveJsonModule": true,
    "types": [
      "node",
      "jest"
    ],
  },
  "exclude": [
    "node_modules",
    "**/*.test.ts"
  ]
}

Our webpack.config.ts for the function (note that this file is TypeScript as well):

const path = require('path');
import webpack from 'webpack';

const env = process.env.ENV ? process.env.ENV : "dev"

const config: webpack.Configuration = {
  mode: env === 'main' ? 'production' : 'development',
  // devtool: 'source-map',
  entry: './lib/index.ts',
  target: 'node',
  node: {
    __dirname: true,
  },
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        use: 'ts-loader',
        exclude: /node_modules/,
      },
    ],
  },
  resolve: {
    extensions: [ '.tsx', '.ts', '.js' ],
    modules: ["node_modules"]
  },
  output: {
    filename: 'index.js',
    libraryTarget: 'commonjs',
    path: path.resolve(__dirname, 'src'),
  }
};

export default config 

The source files live in the /lib directory in the function's folder, while the compiled output is placed into /src.

[lucajung]

+1

[OperationalFallacy]

@r0zar has a point.

I personally develop all my backend code as an independent typescript node package. You can then use all the things you like such as jest for unit testing. The lambda then installs that private node package and becomes a very thin javascript layer on top of that node package built in typescript.

This has worked out well and lets us develop and test things independently of the amplify workflow. The package also becomes very re-usable for scenarios outside of amplify (such as our own cli to interact with our backend).

That's an interesting approach. How does it work?

[hisham]

@r0zar has a point.
I personally develop all my backend code as an independent typescript node package. You can then use all the things you like such as jest for unit testing. The lambda then installs that private node package and becomes a very thin javascript layer on top of that node package built in typescript.
This has worked out well and lets us develop and test things independently of the amplify workflow. The package also becomes very re-usable for scenarios outside of amplify (such as our own cli to interact with our backend).

That's an interesting approach. How does it work?

It works just as I described. Develop your backend code as an independent npm package, that is installed via a lambda layer with this package in the layer's package.json.

[OperationalFallacy]

@r0zar has a point.
I personally develop all my backend code as an independent typescript node package. You can then use all the things you like such as jest for unit testing. The lambda then installs that private node package and becomes a very thin javascript layer on top of that node package built in typescript.
This has worked out well and lets us develop and test things independently of the amplify workflow. The package also becomes very re-usable for scenarios outside of amplify (such as our own cli to interact with our backend).

That's an interesting approach. How does it work?

It works just as I described. Develop your backend code as an independent npm package, that is installed via a lambda layer with this package in the layer's package.json.

Oh, right, I missed it was "private node package". Makes sense.