@cumulus/cumulus-ecs-task - npm Package Compare versions

Comparing version 1.1.1 to 1.1.2

CHANGELOG.md

@@ -9,2 +9,7 @@ # cumulus-ecs-task change log
 
+## [v1.1.2] - 2018-06-05
+### Fixed
+- **CUMULUS-602** - Format all logs sent to Elastic Search.
+  - Format the error message.
+
 ## [v1.1.1] - 2018-04-23
@@ -28,5 +33,8 @@ ### Fixed
 
-[Unreleased]: https://github.com/cumulus-nasa/cumulus-cumulus-ecs-task/compare/v1.0.2...HEAD
-[v1.0.2]: https://github.com/cumulus-nasa/cumulus-cumulus-ecs-task/compare/v1.0.1...v1.0.2
-[v1.0.1]: https://github.com/cumulus-nasa/cumulus-cumulus-ecs-task/compare/v1.0.0...v1.0.1
+[Unreleased]: https://github.com/cumulus-nasa/cumulus-ecs-task/compare/v1.1.2...HEAD
+[v1.1.2]: https://github.com/cumulus-nasa/cumulus-ecs-task/compare/v1.1.1...v1.1.2
+[v1.1.1]: https://github.com/cumulus-nasa/cumulus-ecs-task/compare/v1.1.0...v1.1.1
+[v1.1.0]: https://github.com/cumulus-nasa/cumulus-ecs-task/compare/v1.0.2...v1.1.0
+[v1.0.2]: https://github.com/cumulus-nasa/cumulus-ecs-task/compare/v1.0.1...v1.0.2
+[v1.0.1]: https://github.com/cumulus-nasa/cumulus-ecs-task/compare/v1.0.0...v1.0.1
 [v1.0.0]: https://github.com/cumulus-nasa/cumulus-ecs-task/tree/v1.0.0

index.js

@@ -18,2 +18,32 @@ 'use strict';
 /**
+ * Constructs JSON to log and logs it
+ * 
+ * @param {string} level - type of log (trace, debug, info, warn, error, fatal)
+ * @param {string} message - message to log
+ * @returns {undefined} - log is printed to stdout, nothing is returned
+ */
+function logMessage(level, message) {
+  const time = new Date();
+  let output = {
+    level,
+    timestamp: time.toISOString(),
+    message
+  };
+
+  console.log(JSON.stringify(output));
+}
+
+/**
+ * 
+ * @param {string} message - message to log
+ * @param {Error} err - Error object
+ * @returns {undefined} - log is printed to stdout, nothing is returned
+ */
+function logError(message, err) {
+   // error stack with newlines and no leading space or tab will result in separate log entry
+   const msg = `${message} ${err.stack.replace(/\n/g, ' ')}`;
+   logMessage('error', msg);
+}
+
+/**
 * Download the zip file of a lambda function from AWS
@@ -83,3 +113,3 @@ *
       if (err) {
-        console.log('error sending heartbeat', err);
+        logError('error sending heartbeat', err);
       }
@@ -104,3 +134,3 @@ }, 60000);
     if (err) {
-      console.log('sendTaskFailure err', err);
+      logError('sendTaskFailure err', err);
     }
@@ -123,3 +153,3 @@ });
     if (err) {
-      console.log('sendTasksuccess error', err);
+      logError('sendTasksuccess error', err);
     }
@@ -249,3 +279,3 @@ });
     poll.on('error', (err) => {
-      console.log('error polling for work with sf.getActivityTask', err);
+      logError('error polling for work with sf.getActivityTask', err);
     });
@@ -252,0 +282,0 @@

package.json

 {
   "name": "@cumulus/cumulus-ecs-task",
-  "version": "1.1.1",
+  "version": "1.1.2",
   "description": "Run lambda functions in ECS",
@@ -5,0 +5,0 @@ "main": "index.js",

README.md

@@ -76,6 +76,19 @@ # cumulus-ecs-task
 
-Here's an example:
+Here's an example deployment configuration that would be placed in app/config.yml:
 
 ```yml
 yourdeployment:
+  # make sure to set these environment variables in the app/.env file
+  params:
+    - name: DockerPassword
+      value: '{{DOCKER_PASSWORD}}'
+    - name: DockerEmail
+      value: '{{DOCKER_EMAIL}}'
+    - name: CmrPassword
+      value: '{{CMR_PASSWORD}}'
+
+  # define the ECS activity
+  activities:
+    - name: EcsTaskHelloWorld
+
   ecs:
@@ -88,10 +101,13 @@ instanceType: t2.small
     docker: 
-      username: cumulususer
+      username: <your docker user name>
     services:
       EcsTaskHelloWorld:
-        image: cumuluss/cumulus-ecs-task:1.0.0
-        cpu: 800
-        memory: 1500
-        count: 0
+        image: cumuluss/cumulus-ecs-task:1.1.1
+        cpu: 500
+        memory: 500
+        count: 0 # increase this to increase the number of tasks
         envs:
+          # env vars needed for core cumulus modules:
+          internal: <name of internal bucket>
+          stackName: <name of deployment (in this case it would be "yourdeployment")>
           AWS_DEFAULT_REGION:
@@ -108,5 +124,2 @@ function: Fn::Sub
             value: EcsTaskHelloWorldLambdaFunction
-
-  activities:
-    - name: EcsTaskHelloWorld
 ```
@@ -117,3 +130,3 @@
 ```
-image: cumuluss/cumulus-ecs-task:1.0.0
+image: cumuluss/cumulus-ecs-task:1.1.1
 ```
@@ -129,3 +142,15 @@
 
+#### Docker credentials
 
+Make sure to set the docker email and password in app/.env:
+
+```
+DOCKER_PASSWORD=<your password>
+DOCKER_EMAIL=<your email>
+```
+
+**Which docker credentials should we use?**
+
+Any valid credentials to authenticate with docker's API service. It's recommended to create an organization-specific docker hub account.
+
 ### IAM permissions
@@ -145,2 +170,3 @@
   - states:GetActivityTask
+  - states:GetExecutionHistory
   Resource: arn:aws:states:*:*:*
@@ -209,6 +235,36 @@ ```
 
+## Troubleshooting
+
+SSH into the ECS container instance.
+
+Make sure the EC2 instance has internet access and is able to pull the image from docker hub by doing:
+
+```
+docker pull cumuluss/cumulus-ecs-task:1.1.1
+```
+
+`cat` the ecs config file to make sure credentials are correct:
+
+```
+cat /etc/ecs/ecs.config
+```
+
+Check if there's multiple entries of the config.
+
+If there is, there are two things to try:
+
+- Delete the ec2 instance and redeploy
+- Delete the incorrect config and restart the ecs agent (I haven't tested this much but I expect it to work. You'll still want to update the docker credentials in the deployment's app directory). Restart the agent by doing:
+  ```sh
+  sudo stop ecs
+  source /etc/ecs/ecs.config
+  sudo start ecs
+  ```
+
 ## Create a release
 
-Bump the version
+To create a release, first make sure the [CHANGELOG.md](CHANGELOG.md) file is updated with all the changes made.
 
+Next, bump the version and the changes will automatically be released upon merge to master.
+
 ```
@@ -215,0 +271,0 @@ npm version <major|minor|patch|specific version>

No alert changes

Improved metrics

Total package byte prevSize

37770

increased by8.79%

Lines of code

399

increased by7.26%

Number of lines in readme file

285

increased by24.45%

No dependency changes