Spark, Hadoop, and Kryo utilities
Classes that implement the Registrar interface can use various shorthands for registering classes with Kryo.
Adapted from RegistrationTest:
register(
cls[A], // comes with an AlsoRegister that loops in other classes
arr[Foo], // register a class and an Array of that class
cls[B] → BSerializer(), // use a custom Serializer
CDRegistrar // register all of another Registrar's registrations
)Serializers and AlsoRegisters are picked up implicitly if not provided explicitly.AlsoRegisters are recursive, allowing for much easier and more robust accountability about what is registered and why, and ensurance that needed registrations aren't overlooked. Configuration: serializable Hadoop-Configuration wrapperContext: SparkContext wrapper that is also a Hadoop Configuration, for unification of "global configuration access" patternsConf: load a SparkConf with settings from file(s) specified in the SPARK_PROPERTIES_FILES environment variableSparkConfBase: trait that brokers setting config key-values and creating a SparkConfKeyPartitioner / Partitioner: shorthands for common Spark-Partitioner-creation patterns
Histogram accumulator