Subject: Re: [mongodb-user] getting writeError
(mongo/util/concurrency/rwlock.h:204) during
massive insert files to mongoDB with gridfs



You might look in the mongod logs to see what it says.  It might be that you are out of some resource on the system.
Asya

On Wed, Aug 9, 2017 at 1:43 AM, <[email protected]> wrote:
Hi, All!
A have a python script witch takes a huge list of images and move them in a circle to mongoDB using gridfs (it stores files in two mongoDB collections, files and chunks).After few thousands of successful transfers I am starting to get following error:
assertion C:\data\mci\77b70aa30ca8170c4ee9bee2ba42cf1f\src\src\mongo/util/concurrency/rwlock.h:204

I tried to debug it and it looks, that it becomes fail during chunk insert, but why this happens I don't know. I test script on my local mongoDB and here is a code on python, which I use
import pymongo
from pymongo import MongoClient
import gridfs

fs = gridfs.GridFS(db)

for image in images_list:
  with open(vers['path'], 'rb') as fileObj:
try:
# try:
try:
grid_in = fs.new_file(filename=image[1]['name'],
metadata={"imageId": image[1]['_id'],
"version": version})
grid_in.write(fileObj)
file_id = grid_in._id
finally:
grid_in.close()

  except pymongo.errors.WriteError as we:
print('%s %s:%s' % (image[1]['_id'], we, vers['path']))
grid_in.abort()
continue

fileObj.closed



I also tried gridfs.put instead of new_file and write and also GridFSBucket.open_upload_stream and GridfsBuscket.upload_from_stream - result always the same. 
Since some moment I am getting error, which a mentioned above

Do you know why this happens and how can I solve it?

Or maybe you can at least answer following question:

- can be some problems because of inserting the same images few times? or images with same filename?
- can it be error because of access to mongoDB (lock mechanizm or smth) and how can I then change it to avoid any conflicts? 

If you have some other ideas, they all are welcome. 

Thank you!

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/797c142a-1015-4fe0-82bb-a51ec8ad22ff%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Asya Kamsky
Lead Product Manager
MongoDB
Download MongoDB - mongodb.org/downloads
We're Hiring! - https://www.mongodb.com/careers

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAOe6dJA9MGQ1bJABs%2Br06wHO4wWf4oY1UoikEKncyhTAUufVoQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.



Programming list archiving by: Enterprise Git Hosting