Subject: [mongodb-user] Re: Inserting the value in
Collection schema in MongoDB from different data
frames using python.

I want to insert this values of each row all together in my mongo db Scheme.

Hi Aman,

Please consider how you’re going to query or retrieve these data later from MongoDB.
I would strongly recommend to review Data Modeling Introduction to see more examples of how documents are modelled.
For example, consider if you have many rows of MAP 101010101 stored into a single document

Any suggestion or some changes I need to do with the code?
uniqueVar_detail = data2.MAP.unique()

I’d like to point out that you’re assuming data2 will have all of the unique MAP id from data1 and data3.

In the code below:

data1 = data1[data1['MAP'] == uniqueVar_detail[i]]
data2 = data2[data2['MAP'] == uniqueVar_detail[i]]
data3 = data3[data3['MAP'] == uniqueVar_detail[i]]

You’re re-assigning the variable data1, data2, and data3 which affects the second iteration of the loop.


The line above is actually converting an object of pandas Series into a string. i.e.

0    1010\n1    1010\nName: map, dtype: int64

“data1” : [{“val1”:str (data1.val1), …

Note that depending on your use case, having an array of subdocuments may create difficulty in querying the data later on. Especially if you’re interested in saving the index of the variable MAP.

Given your provided data example, and without drastic changes to your schema, see an example code below to insert CSV data into MongoDB:

# Add your own error handling as appropriate
import json 

for unique_id in data2.MAP.unique():
    d1 = data1[data1['MAP'] == unique_id]
    d2 = data2[data2['MAP'] == unique_id]
    d3 = data3[data3['MAP'] == unique_id]

    d1_json = json.loads(d1.to_json(orient="index"))
    subdoc1 = {}
    for ix, value in d1_json.iteritems():
        subdoc1[str(ix)] = value

    d2_json = json.loads(d2.to_json(orient="index"))
    subdoc2 = {}
    for ix, value in d2_json.iteritems():
        subdoc2[str(ix)] = value 

    d3_json = json.loads(d3.to_json(orient="index"))
    subdoc3 = {}
    for ix, value in d3_json.iteritems():
        subdoc3[str(ix)] = value 

    db.collectionName.insert_one({"map_message_id": unique_id, 
           "data1": subdoc1, 
           "data2": subdoc2, 
           "data3": subdoc3 })

The content of document MAP 101010101 would look like below :

  "_id": ObjectId("..."),
  "map_message_id": 101010101,
  "data1": {
    "1": {"map": 101010101, "val3": 76, "val2": 71, "val1": 13 },
    "0": {"map": 101010101, "val3": 45, "val2": 78, "val1": 12 }
  "data3": {
    "0": {"map": 101010101, "val9": 9, "val8": 78, "val7": 1 }
  "data2": {
    "1": {"map": 101010101, "val6": 78, "val5": 2353, "val4": 13 },
    "0": {"map": 101010101, "val6": 89, "val5": 7190, "val4": 12 },
    "2": {"map": 101010101, "val6": 65, "val5": 0, "val4": 12 }

See also DataFrame.to_json


You received this message because you are subscribed to the Google Groups "mongodb-user"
For other MongoDB technical support options, see:
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at
To view this discussion on the web visit
For more options, visit

Programming list archiving by: Enterprise Git Hosting