git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Camel lock gets acquired on files but not picked sometime


Also you can use "done" files, eg in the VB script you write a 2nd
empty file with a special name, that tells Camel that the other file
is done being copied and ready.
There is some options on Camel file endpoint you can configure for that also.

On Mon, Aug 6, 2018 at 3:54 PM, Claus Ibsen <claus.ibsen@xxxxxxxxx> wrote:
> Hi
>
> You can configure Camel to only pickup files with a certain file
> pattern, or to exclude names with temporary names, etc.
> There is a number of different filter options, see its doc.
>
> On Mon, Aug 6, 2018 at 3:52 PM, Hemant Pekhale <hemant.pekhale@xxxxxxxxx> wrote:
>> Hi Claus,
>>
>> Thanks for the reply.
>>
>> Yes, I am able to modify the script. But if we do copy the file with
>> temporary name and then rename, I doubt that there may be lock acquired on
>> temporary file name already. because my route has no waiting parameter. So
>> I think it might not be helpful for our case. Please correct me if I am
>> wrong.
>>
>> Regarding the file lock strategies, I think  readLock think
>> readLockCheckInterval think can be tried. Do you have any other suggestion?
>>
>> FYI : Whatever I can try I need to try on Production server so just wanted
>> to tackle this with minimum and required trials.
>>
>> Thanks!
>>
>>
>> On Mon, Aug 6, 2018 at 7:10 PM, Claus Ibsen <claus.ibsen@xxxxxxxxx> wrote:
>>
>>> Hi
>>>
>>> Are you able to modify the VB script so it copies the file using a
>>> temporary name and then rename the file when the copy is done.
>>>
>>> It sounds a bit like the problem is that the VB script is currently
>>> still copying the file, while Camel gets a lock on the file, and
>>> attempt to process it.
>>> File locks is not as easy and it depends a bit what file system, OS,
>>> and how big the files are and other things.
>>>
>>> A good strategy is to copy the files using temporary names or in
>>> another folder, and then do a file rename when the copy is over, that
>>> latter then is an atomic file operation which should not cause file
>>> locking issues.
>>>
>>> You can also try to use some of the other Camel file lock strategies
>>> instead of marker files.
>>> You can see more in the file documentation about these.
>>>
>>>
>>> On Mon, Aug 6, 2018 at 3:19 PM, Hemant Pekhale <hemant.pekhale@xxxxxxxxx>
>>> wrote:
>>> > Hi All,
>>> >
>>> > I have a route which picks files from a folder and do some processing on
>>> it
>>> > and store it in other folder. I have used parameter "delete=true" for
>>> > deleting file from source folder.
>>> >
>>> > Sometime it happens that files are stuck in the source folder and camel
>>> > lock files are also created for all the files but files are neither
>>> process
>>> > nor gets deleted.
>>> >
>>> > There is a VB script which copied files in the source folder. When I
>>> > manually copied the files in the source folder then it never gives issue
>>> > but when the files are copied from VB script then only issue occurs. Also
>>> > the issue does not occur for the first file but it occurs randomly after
>>> > processing number of files.
>>> >
>>> > This happens on server. I tried to reproduce this issue locally but no
>>> luck.
>>> >
>>> > Does anybody have idea about it?
>>> >
>>> > Thanks!
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> -----------------
>>> http://davsclaus.com @davsclaus
>>> Camel in Action 2: https://www.manning.com/ibsen2
>>>
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2



( ! ) Warning: include(msgfooter.php): failed to open stream: No such file or directory in /var/www/git/apache-camel-users/msg03566.html on line 175
Call Stack
#TimeMemoryFunctionLocation
10.0009368696{main}( ).../msg03566.html:0

( ! ) Warning: include(): Failed opening 'msgfooter.php' for inclusion (include_path='.:/var/www/git') in /var/www/git/apache-camel-users/msg03566.html on line 175
Call Stack
#TimeMemoryFunctionLocation
10.0009368696{main}( ).../msg03566.html:0