[OSM-Devserver] proposal for a poor?mans?batch?processing?system ?:)

[Stefan Dettenhofer (StefanDausR)]

Hi Nic,

ok Tuesday is your day!

For your information: I use your bboxSplit - it's great!
I split the whole planet (bz2) in 324 files (cat OSM) with 10°x20° in 
149 minutes!
Then I split the biggest parts (about 70) again with 1°x1° in 79 minutes!

Aio and I use stampfiles, wich are present when we do nothing. They are 
located in  /osm/stampfiles
So we can avoid starting our scripts at the same time. If something goes 
wrong during the timeslots we have an other method for serializing. 
Perhaps you can also create a "nic.stamp" and erase it during you are 
working.


Regards,
Stefan


Nic Roets schrieb:
> Hallo Stefan,
>
> Sometimes I will need to split the whole planet. I think that may take
> a complete 12 hour day that Sven has allocated to us. Especially if
> the server is under other load. And some weeks I would like to test
> new code. So it's better that I have a specific day. Then we don't
> need to change the arrangement when the planet grows.  Let's say
> Tuesdays ?
>
> Regards,
> Nic
>
> On Wed, Apr 28, 2010 at 7:50 AM, Stefan Dettenhofer (StefanDausR)
> <osm at dentro.info> wrote:
>   
>> Hallo Nic,
>>
>> which days do you like? How much time do you need? Perhaps I can start at
>> the same day, when you are ready.
>>
>> I have to do this jobs:
>>
>> - "GermanyPlus" needs about 3h: I would like to start it at all 3 days. I
>> think I could also serialize it with AIO
>>
>> - the Continents (Europe 8h, Africa 3.5h, Asia 8.5h) once a month. So I
>> would start one continent each week
>> - or/and the whole planet needs about 39h: I would split the job so that it
>> can run in several timeslots.
>>
>>
>> @Sven: I think Aio, Nic and I are not the problem. We can serialize our
>> jobs. The problem is the import of the planet in postgres. As long as this
>> import runs, NOTHING else should start.
>>
>>
>> Stefan
>>
>>
>> Nic Roets schrieb:
>>     
>>> Hello Stefan,
>>>
>>> Which of these days are you not using / can you give up ?
>>>
>>> Regards,
>>> Nic
>>>
>>> 2010/4/27 Sven Geggus <lists at fuchsschwanzdomain.de>:
>>> ...
>>>
>>>       
>>>> Tuesday, Thursday and Sunday are then available for other processes
>>>> (8AM-8PM)
>>>>
>>>> Currently you should share these days with StefanD. Just check with
>>>> him how to share these timeslots.
>>>>
>>>>
>>>>         
>>> ...
>>>
>>>
>>>
>>>       
>>
>>
>>     
>
>
>