Spark is a fast and general cluster computing system for Big Data. It provides high-level APIs in Scala, Java, Python, and R, and an optimized engine that supports general computation graphs for data analysis. It also supports a rich set of higher-level tools including Spark SQL for SQL and DataFrames, MLlib for machine learning, GraphX for graph processing, and Spark Streaming for stream processing.
AsySpark is an component of spark, this component can make machine learning work more efficient with a asynchronous computing model.such as asynchronous stochastic gradient descent.
If you want to do something with us, contact us.
1:Dean, NIPS‘13, Li, OSDI‘14 The parameter server architecture 2:淘宝参数服务器架构
1:Langford, NIPS‘09, Agarwal, NIPS‘11 theoretical convergence of asynchronous SGD 2:Li, WSDM‘16 practical considerations for asynchronous SGD with the parameter server
