innovaphone Apps SDK Developer Forum

Registered 5 years 81 days

iqbal Thursday, 26 January 2023, 02:24 PM

Agenda SDK Dialog: App crashes making multiple parallel http requests using http_client lib

Hi I am having issues with:

https://sdk.innovaphone.com/13r2/sdk/common/interface/http_client.htm

when I try to make http requests parallel.

at start of task i check for connection but it still give error:

void TaskHttpRequest::Start(class UTask* user)
{
LOG("TaskHttpRequest::%s \t URL: %s", __FUNCTION__, address.c_str());
this->user = user;
if (!httpClient->Connected())
{
httpClient->Connect(address.c_str());
}
}

app crashes with following error:

01-23 11:34:18.160 ASSERT: Called HTTPClient::Connect() while client is not offline (Connect:198)
[New LWP 31879]
[New LWP 31920]
[New LWP 31921]
[New LWP 31923]
[New LWP 31924]
[New LWP 31925]
[New LWP 32012]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/apps/WhoisOnV2/WhoisOnV2 --manager=/var/run/manager/manager --service-id=whois'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f96921afe8a in raise () from /lib64/libpthread.so.0
[Current thread is 1 (Thread 0x7f96916fe100 (LWP 31879))]

Thread 7 (Thread 0x7f968a7fc700 (LWP 32012)):
#0 0x00007f96921ab77c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00000000004c03c3 in DnsThread::ThreadFunction() ()
#2 0x00000000004bf5da in __dns_thread_main(void*) ()
#3 0x00007f96921a4f3d in start_thread () from /lib64/libpthread.so.0
#4 0x00007f969183b9af in clone () from /lib64/libc.so.6

Thread 6 (Thread 0x7f968affd700 (LWP 31925)):
#0 0x00007f96921ab77c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00000000004c03c3 in DnsThread::ThreadFunction() ()
#2 0x00000000004bf5da in __dns_thread_main(void*) ()
#3 0x00007f96921a4f3d in start_thread () from /lib64/libpthread.so.0
#4 0x00007f969183b9af in clone () from /lib64/libc.so.6

Thread 5 (Thread 0x7f968b7fe700 (LWP 31924)):
#0 0x00007f96921ab77c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00000000004c03c3 in DnsThread::ThreadFunction() ()
#2 0x00000000004bf5da in __dns_thread_main(void*) ()
#3 0x00007f96921a4f3d in start_thread () from /lib64/libpthread.so.0
#4 0x00007f969183b9af in clone () from /lib64/libc.so.6

Thread 4 (Thread 0x7f968bfff700 (LWP 31923)):
#0 0x00007f96921aec42 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007f96921a77e1 in pthread_mutex_lock () from /lib64/libpthread.so.0
#2 0x00000000004c032b in DnsThread::ThreadFunction() ()
#3 0x00000000004bf5da in __dns_thread_main(void*) ()
#4 0x00007f96921a4f3d in start_thread () from /lib64/libpthread.so.0
#5 0x00007f969183b9af in clone () from /lib64/libc.so.6

Thread 3 (Thread 0x7f9690ec3700 (LWP 31921)):
#0 0x00007f96921ab77c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00000000004c03c3 in DnsThread::ThreadFunction() ()
#2 0x00000000004bf5da in __dns_thread_main(void*) ()
#3 0x00007f96921a4f3d in start_thread () from /lib64/libpthread.so.0
#4 0x00007f969183b9af in clone () from /lib64/libc.so.6

Thread 2 (Thread 0x7f96916fa700 (LWP 31920)):
#0 0x00007f96921ab77c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00000000004c03c3 in DnsThread::ThreadFunction() ()
#2 0x00000000004bf5da in __dns_thread_main(void*) ()
#3 0x00007f96921a4f3d in start_thread () from /lib64/libpthread.so.0
#4 0x00007f969183b9af in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7f96916fe100 (LWP 31879)):
#0 0x00007f96921afe8a in raise () from /lib64/libpthread.so.0
#1 0x00000000004807d8 in HTTPClient::Connect(char const*, char const*, char const*, int) ()
#2 0x000000000040bfdd in WhoisOn::PbxPresenceUpdated(std::vector<UserPresence, std::allocator<UserPresence> > const&) ()
#3 0x000000000040d30d in WhoisOn::PbxPresenceUpdated(char const*, IPbxApiPresence*) ()
#4 0x000000000048c289 in PbxApiTaskPresenceSubscription::Message(json_io&, unsigned short, char const*, char const*) ()
#5 0x00000000004617b3 in AppWebsocket::WebsocketRecvResult(IWebsocket*, void*, unsigned long, bool, bool) ()
#6 0x00000000004aace9 in WebserverWebsocketHandler::HandleMessage(IWebserverMessage*, void*, unsigned long) ()
#7 0x00000000004a2779 in WebserverPlugin::SocketRecvResult(ISocket*, void*, unsigned long) ()
#8 0x000000000047ea0b in UDSocket::ReceiveData() ()
#9 0x000000000047eb4d in UDSocket::IoExec(void*) ()
#10 0x000000000045b259 in IoMux::ExecListRun() ()
#11 0x000000000045b6cc in IoMux::Run() ()
#12 0x0000000000409181 in main ()

Moderator

Registered 14 years 345 days

Daniel Deterding (innovaphone) Friday, 27 January 2023, 07:03 AM

Re: Agenda SDK Dialog: App crashes making multiple parallel http requests using http_client lib

Hi Iqbal,

Connected() currently just returns the socket connection state. This might be false but the HttpClient can be already trying to connect to a socket, because of a previous Connect() call which hasn't been finished yet.

So this function is not suitable to determine if an instance can be reused. I will change the documentation to reflect this fact.

You have to handle the logic yourself right now: an HttpClient instance is ready to be reused if:

you received a HTTPClientShutdown
you received a HTTPClientRecvResult with transferComplete == true (then the socket is still connected and you may call SetRequestType again for a reuse of the same TCP connection)

Greetings,
Daniel

Registered 5 years 81 days

iqbal Friday, 27 January 2023, 09:09 AM

Re: Agenda SDK Dialog: App crashes making multiple parallel http requests using http_client lib

Hi Daniel

if I have to wait for last http Request to complete,

then how can I make multiple requests parallely. In my situation I have

std::vector<unique_ptr<TaskHttpRequest>> httpTasksVec.

I fill that vector on some event and start all vector Tasks in for loop.

after http call is successful or failed I remove that TaskHttpRequest instance from std::vector<unique_ptr<TaskHttpRequest>> httpTasksVec.

as in Logs you can see every TaskHttpRequest owns its own httpClient from sdk.

in this situation I dont understand why call of

httpClient->Connect(address.c_str());

from one instance can effect call of

httpClient->Connect(address.c_str());

from other instances. is there something that i should consider that is not described in documentation.

Best Regards

Iqbal

Moderator

Registered 14 years 345 days

Daniel Deterding (innovaphone) Friday, 27 January 2023, 10:03 AM

Re: Agenda SDK Dialog: App crashes making multiple parallel http requests using http_client lib

Hi Iqbal,

every call to IHTTPClient::Create returns a single instance of IHTTPClient which can be just used for one task which handles this single HTTP request.

You can of course call IHTTPClient::Create within every task once to create a new IHTTPClient instance. Then you have parallel requests.

But keep in mind, that a few outgoing parallel requests won't hurt but a whole lot of requests will consume resources (network bandwith, RAM etc.) and if these requests are done on the same server, you may get blocked by its firewall/RP for too many requests per timespan.

So you may need a queue to just process X requests at the same time. For example the Devices App just performs 20 requests at the same time and queues further requests to avoid overload situations.

Greetings,
Daniel

Registered 5 years 81 days

iqbal Tuesday, 31 January 2023, 03:08 PM

Re: Agenda SDK Dialog: App crashes making multiple parallel http requests using http_client lib

Any update about sample code. please let me know if you can provide sample code for parallel request usecase. As I need it urgently.

Iqbal

Moderator

Registered 12 years 277 days

Andreas Fink (innovaphone) Tuesday, 31 January 2023, 07:18 PM

1 of 1 users consider this post helpful

Re: Agenda SDK Dialog: App crashes making multiple parallel http requests using http_client lib

Hello Iqbal,

please find attached an example Class HttpRequest that utilises HTTPClient Library which implements Task pattern.

It is possible to instantiate multiple objects of the HttpRequest by HttpRequest req = new HttpRequest(service, url, nullptr, false); and start each request by calling req->Start(this);

This will open an own socket for each request and is suitable for requests to different hosts. In case all your requests are directed to the same host and only different method/path is used for each request, you can reuse the HTTPClient object. After receiving HTTPClientRecvResult with transferComplete=true you can call SetRequestType again on the same IHTTPClient object, but provide other path as resourceName parameter.

Best Regards

Andreas Fink

http_request.zip

Registered 5 years 81 days

iqbal Wednesday, 1 February 2023, 09:12 AM

Re: Agenda SDK Dialog: App crashes making multiple parallel http requests using http_client lib

Hi Andreas

Thanks for sample code.

"This will open an own socket for each request and is suitable for requests to different hosts. In case all your requests are directed to the same host and only different method/path is used for each request, you can reuse the HTTPClient object. After receiving HTTPClientRecvResult with transferComplete=true you can call SetRequestType again on the same IHTTPClient object, but provide other path as resourceName parameter."

that means i have to wait for previous http request for completion if i am using same host. In my situation host is same and i want to make http requests to that host simultaneously.

Iqbal

Moderator

Registered 12 years 277 days

Andreas Fink (innovaphone) Wednesday, 1 February 2023, 02:53 PM

Re: Agenda SDK Dialog: App crashes making multiple parallel http requests using http_client lib

Hello Iqbal,

how many parallel requests are intended to be maintained in your case?

You have to instantiate multiple HttpRequest objects, in case you want to maintain parallel requests over multiple sockets.

Best Regards

Andreas Fink

Registered 5 years 81 days

iqbal Thursday, 2 February 2023, 12:21 PM

Re: Agenda SDK Dialog: App crashes making multiple parallel http requests using http_client lib

Andreas

"how many parallel requests are intended to be maintained in your case?"

i need probably 5-10 parallel request to same host.

"You have to instantiate multiple HttpRequest objects"

I am already instantiating multiple HttpRequest objects

first request goes well and second request throws:

01-23 11:34:18.160 ASSERT: Called HTTPClient::Connect() while client is not offline (Connect:198)

I assume there is only one connection in background. calling this shared connection to different multiple HttpRequest objects for same host give this error.

Could you please check this in sdk implementation.

Iqbal

Moderator

Registered 12 years 277 days

Andreas Fink (innovaphone) Thursday, 2 February 2023, 12:29 PM

Re: Agenda SDK Dialog: App crashes making multiple parallel http requests using http_client lib

Hello Iqbal,

I'm not sure we are talking about the same HttpRequest Class implementation. I have attached a file http_request.zip in one of my previous messages, that implements own sockets for every instance of the HttpRequest object. Are you using this implementation now, and still get the same assert?

Best Regards

Andreas Fink

Registered 5 years 81 days

iqbal Thursday, 2 February 2023, 01:08 PM

Re: Agenda SDK Dialog: App crashes making multiple parallel http requests using http_client lib

Hi Andreas

I was not able to test your Code. But I compared it with my own code. in httpClient part they both are same. so I didnt checked sample code in isolation.

I will check it in Isolation and let you know.

Iqbal

Moderator

Registered 12 years 277 days

Andreas Fink (innovaphone) Thursday, 2 February 2023, 01:18 PM

1 of 1 users consider this post helpful

Re: Agenda SDK Dialog: App crashes making multiple parallel http requests using http_client lib

Hi Iqbal,

than i would also double check, if the same instance of your class is used twice.

Best Regards

Andreas Fink

You are here