Post has attachment
OmniThreadLibrary 3.07.6 is out with official support for Delphi 10.3 Rio!

Just a quick note - last OTL release compiles and works fine in Rio.

There will be a new release with some bugfixes and official Rio support in few days.

Anyone have insights how to successfully run Delphi in debug mode when using TBBMalloc. I'm running multiple task in a Join and when I execute it proceeds but the threads don't appear to be doing any processing. If I run normally in runtime mode it does chug along but at a certain point I get an out of memory crash. That's what I'm trying to find and see if I can stem any leakage. One other thing I noticed when I run with less stressful work I can debug and find that even though I set the Join to nil, the threads remain listed as if the Join is still available. Any help would be very much appreciated.

I'm trying to find out why my Form is not responding. I have a simple VCL Form with a label and a progress bar for reporting purposes. I have 1000 tasks that I add to a Join in groups of 4 to match my processors. The processing proceeds fine and after each batch of 4, I simply update the label and progress bar from the main thread. If I don't touch the form everything proceeds fine to completion but if I do click on the form the it displays not responding. The funny thing is if I wait for the normal amount of time then it always completes and the UI is finally updated. I'm also executing the Join with NoWait and then after that I have a simple while statement that does the WaitFor(INFINITE). Once again I'm not in a position to post code but I think my description above is very straightforward. I would really appreciate it if anyone could suggest a simple answer - thanks, Aidan

Post has attachment
It's moving time!

As the "Delphi Developers" community, OmniThreadLibrary support forum will move from Google+ to https://en.delphipraxis.net.

A separate sub-forum was created there for OTL-specific discussions: https://en.delphipraxis.net/forum/32-omnithreadlibrary/

I'll probably be present here until Google shuts G+ down, so no hurry :)

Post has attachment
I’m using a logging technique described in:

https://stackoverflow.com/questions/8134526/implement-thread-safe-logging

I really like the elegance of the solution. However, I’m running into spurious and intermittent problem when thousands of messages are rapidly logged. It appears that TOmniResourceCount.Handle is set but never reset. The result is this bit of code:

while DSiWaitForTwoObjects(task.TerminateEvent, FLogMsgCount.Handle, false, CMaxLogTimeout_ms) <> WAIT_OBJECT_0 do
Flush;

Continuously executes and pegs one of the CPUs to 100%.

I’ve stared at the code of TOmniResourceCount and I think I have a scenario where this can occur.

Referencing TOmniResourceCount in OtlSync. Given that:

orcNumResources is 1.
TOmniResourceCount.Allocate is called by thread A.
TOmniResouceCount.Release is called by thread B.

Thread A runs first. It acquires a lock in Allocate, which causes Thread B to block in Release.

This code in Allocate is eventually processed:

if orcNumResources.Value > 0 then
begin
Result := true;
resourceCount := cardinal(orcNumResources.Decrement);
if resourceCount = 0 then begin
orcLock.Release; //prevent race condition - another thread may wait on orcHandle and destroy this instance
ResetEvent(orcAvailable);
SetEvent(orcHandle);
Exit; // skip final Release
end;
break; //repeat
end;

resourceCount is 0 so orcLock.Release is called. At this point, before ResetEvent(orcAvailable) runs, a thread switch occurs. Thread B is now free to run the Release code.

orcLock.Acquire;
try
Result := cardinal(orcNumResources.Increment);
if Result = 1 then begin
ResetEvent(orcHandle);
SetEvent(orcAvailable);
end;
finally orcLock.Release; end;

This code completes. Perhaps a few more Releases are run since the while loop is tight. This would put orcNumResources well past 1. Then we switch back to the first thread that is still in Allocate and run:

ResetEvent(orcAvailable);
SetEvent(orcHandle);
Exit; // skip final Release

orcHandle is set. It may remain set for a long time before enough Allocates and Releases occur to call Reset(orcHandle) once more. This results in the while loop above to always evaluate to true and process Flush.

Does this scenario make sense? If so, is the solution modifying Allocate from:

if resourceCount = 0 then begin
orcLock.Release; //prevent race condition - another thread may wait on orcHandle and destroy this instance
ResetEvent(orcAvailable);
SetEvent(orcHandle);
Exit; // skip final Release
end;

to:

if resourceCount = 0 then begin
ResetEvent(orcAvailable);
SetEvent(orcHandle);
Break; // call final Release
end;

I am trying "Observing lock-free collections" with "TOmniQueue", In my code observer not raising any event, every time I Enqueue message. Am I missing anything here?

Below is my code.

procedure TfrmTestOmniQueue.FormCreate(Sender: TObject);
begin
FBlockSize := 64;
MsgQueue := TOmniQueue.Create(FBlockSize * 1024);
Observer := CreateContainerWindowsEventObserver;
MsgQueue.ContainerSubject.Attach(Observer, coiNotifyOnAllInserts);
Observer.Activate;

Task := CreateTask(OnNewMes);
Task.OnMessage(ReportMessage).WaitFor(Observer.GetEvent);
end;

procedure TfrmTestOmniQueue.FormDestroy(Sender: TObject);
begin
if assigned(MsgQueue) and assigned(Observer) then
MsgQueue.ContainerSubject.Detach(Observer, coiNotifyOnAllInserts);

FreeAndNil(Observer);
MsgQueue.Free;

if Assigned(Task) then
begin
Task.Terminate;
Task := nil;
end;
end;

procedure TfrmTestOmniQueue.OnNewMesssage(const Task: IOmniTask);
var
value: TOmniValue;
begin
while MsgQueue.TryDequeue(value) do
lbLog.Items.Add(value);
end;

procedure TfrmTestOmniQueue.btnSetMessageClick(Sender: TObject);
begin
MsgQueue.Enqueue(FormatDateTime('dd.mm.yy hh.nn.ss.zzz', Now));
end;

I’m getting a Queue is full exception. What is the limit on a queue? Is there a way to control this?

Post has attachment
I want to try something similar to your "Web download and database storage" example, but the first stage will be querying tables and providing results to the second stage for database storage. I need a connection pool for use in the first stage's "pipelineStage" Retriever procedure. It looks like "Stage" can accept a second optional "taskConfig" parameter.

1. Would this parameter be the place to define a configuration for the Retriever Stage, specifying a ThreadPool (Connection Pool) for the Retriever?

2. Or would a "Background Worker" or some other approach be better than a pipeline for this job?

What are the main reasons why a join with the same number of tasks would end up running at the same pace as simply running the work sequentially. In my code I have for example 1000 cases to process and I break them up on the basis of the number of logical processors. So the breakdown happens 250 times adding the procedure to the join 4 at a time. When executed it looks like there are 4 threads processing but the overall completion time is around 7.5 seconds versus 9.4 seconds when I just walk through the 1000 cases one by one. Any thoughts/help would be very much appreciated - cheers, Aidan.
Wait while more posts are being loaded