I couldn't find a complete Scala version using mongo-hadoop v1.3.1 to read a mongodump bson file, so here's one I prepared earlier:
val bsonData = sc.newAPIHadoopFile(
"file:///your/file.bson",
classOf[com.mongodb.hadoop.BSONFileInputFormat].asSubclass(classOf[org.apache.hadoop.mapreduce.lib.input.FileInputFormat[Object, org.bson.BSONObject]]),
classOf[Object],
classOf[org.bson.BSONObject])
Note that (for v1.3.1) we need to subclass com.mongodb.hadoop.BSONFileInputFormat to avoid this compilation error: "inferred type arguments do not conform to method newAPIHadoopFile's type parameter bounds". This isn't required if reading from Mongo directly using com.mongodb.hadoop.MongoInputFormat.
Also, you can pass a Configuration object as a final parameter if you need to set any specific conf values.
For more bson examples see here: https://github.com/mongodb/mongo-hadoop/blob/master/BSON_README.md
For Java examples see here: http://crcsmnky.github.io/2014/07/13/mongodb-spark-input/
Showing posts with label mongo. Show all posts
Showing posts with label mongo. Show all posts
Tuesday, January 27, 2015
Wednesday, November 28, 2012
Decode a mongo BinData field into a Guid with C#
This C# snippet will decode a mongo BinData field into a Guid
got this from Mongo:
{
"_id" : ObjectId("50b40a5db14ea8902cd5d5ed"),
"MyGuid : new BinData(3, "MC/0P9nkEeGgjYQrK2V3pQ=="),
...
}
but want to know the value of MyGid? Try this little snippet:
using System;
namespace MongoGuidDecoder
{
class Program
{
static void Main(string[] args)
{
var input = args[0];
var bytes = Convert.FromBase64String(input.Trim());
var guid = new Guid(bytes);
Console.WriteLine(input);
Console.WriteLine(guid);
}
}
}
Now run to get the Guid representation:
>MongoGuidDecoder MC/0P9nkEeGgjYQrK2V3pQ==
MC/0P9nkEeGgjYQrK2V3pQ==
3ff42f30-e4d9-e111-a08d-842b2b6577a5
got this from Mongo:
{
"_id" : ObjectId("50b40a5db14ea8902cd5d5ed"),
"MyGuid : new BinData(3, "MC/0P9nkEeGgjYQrK2V3pQ=="),
...
}
but want to know the value of MyGid? Try this little snippet:
using System;
namespace MongoGuidDecoder
{
class Program
{
static void Main(string[] args)
{
var input = args[0];
var bytes = Convert.FromBase64String(input.Trim());
var guid = new Guid(bytes);
Console.WriteLine(input);
Console.WriteLine(guid);
}
}
}
Now run to get the Guid representation:
>MongoGuidDecoder MC/0P9nkEeGgjYQrK2V3pQ==
MC/0P9nkEeGgjYQrK2V3pQ==
3ff42f30-e4d9-e111-a08d-842b2b6577a5
Subscribe to:
Posts (Atom)