Media Insights Engine (MIE) is a framework to accelerate the development of serverless applications that process video, images, audio, and text with artificial intelligence services and multimedia services on AWS. MIE is most often used to:
MIE includes a demo GUI for video content analysis and search. The Implementation Guide explains how to build other applications with MIE.
The Media Insights sample application lets you upload videos, images, audio and text files for content analysis and add the results to a collection that can be searched to find media that has attributes you are looking for. It runs an MIE workflow that extracts insights using many of the ML content analysis services available on AWS and stores them in a search engine for easy exploration. A web based GUI is used to search and visualize the resulting data along-side the input media. The analysis and transformations included in MIE workflow for this application include:
Data are stored in Amazon Elasticsearch Service and can be retrieved using Lucene queries in the Collection view search page.
You can deploy MIE and the demo GUI in your AWS account with the following instructions:
| Region | Launch |
|---|---|
| US East (N. Virginia) |  |
| US West (Oregon) | |
Stack name and AdminEmail.You can view the status of the stack in the AWS CloudFormation Console in the Status column. You should see a status of CREATE_COMPLETE in approximately 15-30 minutes.
For more information about stack deployment, see the section on installation parameters.
To access the web application, you will need the email address you provided in the AWS CloudFormation template, a temporary password, which is emailed to your specified address, and the URL of your deployed MIE sample application.
In the same email that includes your temp password, there is a link to the MIE stack overview.
Outputs tab of the MIE stack.MediaInsightsWebAppUrl output. This is the URL you will use to access the sample application.MediaInsightsWebAppUrl value into the navigation bar and access the app.Upload link in the navigation header and explore the application.After the stack successfully deploys, you can find important interface resources in the Outputs tab of the MIE CloudFormation stack.
MediaInsightsWebAppUrl is the Url to access sample Media Insights web application
DataplaneApiEndpoint is the endpoint for accessing dataplane APIs to create, update, delete and retrieve media assets
DataplaneBucket is the S3 bucket used to store derived media (derived assets) and raw analysis metadata created by MIE workflows.
ElasticsearchEndpoint is the endpoint of the Elasticsearch cluster used to store analysis metadata for search
MediaInsightsEnginePython37Layer is a lambda layer required to build new operator lambdas
WorkflowApiEndpoint is the endpoint for accessing the Workflow APIs to create, update, delete and execute MIE workflows.
WorkflowCustomResourceArn is the custom resource that can be used to create MIE workflows in CloudFormation scripts
MIE itself does not have a significant cost footprint. The MIE control plane and data plane generally cost less than $1 per month. However, when people talk about the cost of MIE they're generally talking about the cost of running some specific application that was built on top of MIE. Because those costs can vary widely you will need to get pricing information from the documentation for those applications. As a point of reference, see the README for the Content Analysis application that is included under the webapp directory.
The latest MIE release has been verified to support videos up to 2 hours in duration.
Media Insights Engine is a serverless architecture on AWS. The following diagram is an overview of the major components of MIE and how they interact when an MIE workflow is executed.
Triggers the execution of a workflow. Also triggers create, update and delete workflows and operators. Monitors the status of workflows.
Executes the AWS Step Functions state machine for the workflow against the provided input. Workflow state machines are generated from MIE operators. As operators within the state machine are executed, the interact with the MIE data plane to store and retrieve derived asset and metadata generated from the workflow.
Generated state machines that perform media analysis or transformation operation.
Generated state machines that execute a number of operators in sequence.
Stores media assets and their associated metadata that are generated by workflows.
Trigger create, update, delete and retrieval of media assets and their associated metadata.
Stores metadata for an asset that can be retrieved as a single block or pages of data using the objects AssetId and Metadata type. Writing data to the pipeline triggers a copy of the data to be stored in a Kinesis Stream.
A lambda function that consumes data from the data plane pipeline and stores it (or acts on it) in another downstream data store. Data can be stored in different kind of data stores to fit the data management and query needs of the application. There can be 0 or more pipeline consumers in a MIE application.
You can deploy MIE and the demo GUI in your AWS account with the one-click deploy buttons shown above.
Stack Name: Name of stack. Defaults to mie.
System Configuration
10.Operators
true.Workflows
false.false.true.false.true.Sample Applications
true.Other parameters
true.MIE is a reusable architecture that can support many different applications. Examples:
Join our Gitter chat at https://gitter.im/awslabs/aws-media-insights-engine. This public chat forum was created to foster communication between MIE developers worldwide.
MIE is built to be extended in the following ways:
See the Implementation Guide for the MIE API reference and builder's guide.
Visit the Issue page in this repository for known issues and feature requests.
See the CONTRIBUTING file for how to contribute.
The MIE logo features a clapperboard representing multimedia, centered inside a crosshair representing under scrutiny.
See the LICENSE file for our project's licensing.
Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.