[Http] Implement sceHttp library #18003
Conversation
|
Btw how do i clean up temporary/intermediate files that was used during |
|
Yu-Gi-Oh! GX: Tag Force 2 What am I doing wrong? |
|
The unimplemented The menu you're trying to open is probably the "Go to the Yu-Gi-Oh! website" which supposed to open a web browser/view through Sorry, i forgot that Yu-Gi-Oh uses |
Basically just remove build/CMakeFiles iirc. Or you can just remove all of build/. -[Unknown] |
Common/Net/HTTPClient.cpp
Outdated
| @@ -329,7 +329,7 @@ int Client::SendRequestWithData(const char *method, const RequestParams &req, co | |||
| return 0; | |||
| } | |||
|
|
|||
| int Client::ReadResponseHeaders(net::Buffer *readbuf, std::vector<std::string> &responseHeaders, net::RequestProgress *progress) { | |||
| int Client::ReadResponseHeaders(net::Buffer *readbuf, std::vector<std::string> &responseHeaders, net::RequestProgress *progress, std::string * httpCode) { | |||
There was a problem hiding this comment.
nit: Would prefer statusLine rather than httpCode which isn't very clear. Also would prefer * next to the name, as is the convention used elsewhere. It's nice to dream that C+ would make pointer part of the type, but it isn't. int* foo, bar is not two pointers, so the * is objectively in the wrong place.
-[Unknown]
There was a problem hiding this comment.
i think VS2022 inserted that space when i copy-paste it while fixing the conflict during merge 🤔 because i never put spaces on both sides of * except for multiplication, it supposed to be *httpCode and missed the changes done by VS, but yeah the naming is odd when i read it again😅
There was a problem hiding this comment.
Yeah, VS2022 seems to have some weird default formatting settings, like that ugly else formatting...
-[Unknown]
Core/HLE/sceHttp.cpp
Outdated
| const std::string& s = httpLine_ + delim + imploded.str(); | ||
| u32 sz = (u32)s.size(); | ||
|
|
||
| u32* headerAddr = (u32*)Memory::GetPointerUnchecked(headerAddrPtr); |
There was a problem hiding this comment.
Should validate these pointers and at least not crash...
-[Unknown]
Core/HLE/sceHttp.cpp
Outdated
|
|
||
| entityLength_ = -1; | ||
| for (std::string& line : responseHeaders_) { | ||
| if (startsWithNoCase(line, "Content-Length:")) { |
There was a problem hiding this comment.
It's valid to use Content-Length : 42 FWIW.
-[Unknown]
|
|
||
| std::lock_guard<std::mutex> guard(httpLock); | ||
| httpObjects.clear(); | ||
| // Reserve at least 1 element to prevent ::begin() from returning null when no element has been added yet |
There was a problem hiding this comment.
That's not really what reserve does? You'd need to resize(1) for that. reserve just makes sure it's ready to have that much memory. I don't know if an empty container is actually required to have a valid pointer return for [0].
-[Unknown]
There was a problem hiding this comment.
resize(1) will make count/size resulting to 1 isn't?
i only need to allocate the space without increasing the number of elements, so begin() can return an iterator object which matches end() when empty, without it apparently returned null instead of an iterator object.
The reason it must not be null is because on some part of the code the iterator object being increased using +1 or +id i think (i haven't re-read the whole code while fixing the conflict), thus could cause access violation.
So i'm using reserve just to ensure that begin() on an empty vector will always returns a valid iterator object where i can use + index_number without problem (as long not exceeding the number of elements).
And as i remembered it doesn't have this issue when the httpObjects is freshly created (ie. begin() of an empty vector returns an iteration object as in most c++ examples), only happened when reused i think (or was it the opposite, i kinda forgot😅 was getting a crash because of that null), may be i erased the elements the wrong way🤔 most of the one-liner code that dealt with vectors was taken from stackoverflow 😅
There was a problem hiding this comment.
Hm, we should really should have a bool then or something. Trying to rely on whether there's memory at &x[0] when empty just seems like asking for trouble.
-[Unknown]
|
With this PR applied it lets LBP PSP get further in its networking, although it still does not send any requests to a custom server. here is the log after i try to connect from LBP PSP |
Which menu did you choose to for this logs to appear? Also, do you know any private server i can test it with? |
I chose "go online" from the community moon, and even with faked sceNp, it should at least attempt to make a connection to a server, all you need to do is decrypt the EBOOT.BIN, patch the 2 URLs (search for http: in the EBOOT) to There is no functional LBP PSP private server right now, thats what i am trying to work on, but the real hardware is not cooperating update: i used the most recent version of the game, and after patching i get this message Even though the URL is 100% valid |
You will need the test build from here #14256 (comment) which have a lot of the sceNetInet* syscalls being implemented for infrastructure support. PS: Those test builds were based on an older version of PPSSPP, so you will need to use a matching assests files if it only came with the binary file. |
I was testing on this PR rebased to master, but that branch doesnt seem to get any further, With this PR i was at least getting to the URI parasing part (see my edit to this comment), but this other infrastructure branch seems to fail way earlier |
It seems to came from 2nd argument from behind, it will get invalid args if the value is not a valid address, i guess it's allow null ptr O.o may be not expecting any output. |
|
dirty_patch.txt |
Oops, those test build probably didn't have this PR included yet, although the one i had on my laptop already included it but i haven't update the test build yet (due to low free space i can only build the Windows version). |
If theres an open branch to pull from i can do the rebase/build myself |
Based on the logs, it probably expecting an output data from |
That makes sense, i'll just stub it out and see what happens |
|
Stubbing out a bunch of unimplemented things to just return It seems to be calling into I believe what its trying to do is use |
It's currently a pile of stashed code as i haven't had the time to separate it into smaller commits, or at least separate it by library as things can get tangled each other (especially the constant/definitions) |
I would very appreciate if you were to push that stash to a separate branch as just a single "big commit" when/if you get the time. I can work out how to get it working on my machine, and im willing to implement anything more missing to get LBP PSP at least making requests to a server, and i'll send any patches i make over (i mainly just dont want to duplicate any existing work as i poke at this to get it going) |
|
Using these patches half-stable-patch.txt i am able to semi-consistently get in-game and connect to a custom server, if you would like more details on how to setup said custom server for testing, you can find me on discord with the username explanation of the patches: HTTPClient.cpp: Make all socket work synchronous, without this all requests that would get out would send no data and just silently disconnect, giving the "connection refused" errno remaining bugs: the client loves to send extra garbage data after requests, and the game loves to crash trying to access address 0x0 (this is maybe caused by a half-implemented server, but its hard to be 100% certain) |
|
What kind of crash did you get? game crash (Blue/Purple screen on PPSSPP) or PPSSPP crash (in case another issue where current thread is null)? For // Address families
#define PSP_NET_INET_AF_UNSPEC 0 // unspecified
#define PSP_NET_INET_AF_LOCAL 1 // local to host (pipes, portals)
#define PSP_NET_INET_AF_UNIX PSP_NET_INET_AF_LOCAL // backward compatibility
#define PSP_NET_INET_AF_INET 2 // internetwork: UDP, TCP, etc.
int convertSocketDomainPSP2Host(int domain) {
switch (domain) {
case PSP_NET_INET_AF_UNSPEC:
return AF_UNSPEC;
case PSP_NET_INET_AF_LOCAL:
return AF_UNIX;
case PSP_NET_INET_AF_INET:
return AF_INET;
}
return hleLogError(SCENET, domain, "Unknown Socket Domain");
}
// TODO: Need to find out whether it's possible to get partial output or not, since Coded Arms Contagion is using a small bufsize(4)
static u32 sceNetInetInetNtop(int af, u32 srcInAddrPtr, u32 dstBufPtr, u32 bufsize) {
WARN_LOG(SCENET, "UNTESTED sceNetInetInetNtop(%i, %08x, %08x, %d)", af, srcInAddrPtr, dstBufPtr, bufsize);
if (!Memory::IsValidAddress(srcInAddrPtr)) {
return hleLogError(SCENET, 0, "invalid arg");
}
if (!Memory::IsValidAddress(dstBufPtr) || bufsize < 1/*8*/) { // usually 8 or 16, but Coded Arms Contagion is using bufsize = 4
inetLastErrno = ENOSPC;
return hleLogError(SCENET, 0, "invalid arg");
}
if (inet_ntop(convertSocketDomainPSP2Host(af), Memory::GetCharPointer(srcInAddrPtr), (char*)Memory::GetCharPointer(dstBufPtr), bufsize) == NULL) {
//return hleLogDebug(SCENET, 0, "invalid arg?"); // Temporarily commented out in case it's allowed to have partial output
}
return hleLogSuccessX(SCENET, dstBufPtr, "%s", safe_string(Memory::GetCharPointer(dstBufPtr)));
}I also have the resolvers library working (which i use on my test builds), but i won't include them in this PR as it's a separate library (will create a different PR for it later) For |
|
Hmm.. based on this logs i'm getting: The input URL without the host domain, should be valid if there is a "HOST" header, and should probably constructed into a fully qualified URI using the information used in Anyway, since LBP is using HTTPS, it won't be covered on this PR (may be until i replaced HTTPClient with naett) |
many many crashes like this |
I have a game patch which forces HTTP, which is how im doing testing |
|
The blue screen i got always have unknown module/syscall occurrence, so it probably related to that module/syscall PS: Trying to reset the game (through Emulation Menu) while in this blue screen seems to cause Invalid access flooding in the logs and locks up unable to do anything until... PPSSPP crashed (when running PPSSPP through VS2022) |
|
The latest patch did not seem to help, as it looks like the game is sending in a blank string for |
|
Is there any other syscall before sceHttpCreateConnection that shows a valid domain? |
aha, i had forgotten to replace 0.0.0.0 feels wrong, but it seems to connect fine |
|
And i did just confirm that removing |
|
The delay itself isn't much, but it could trigger another PSP thread to run, and if that other thread also use HTTPClient to download something else .. it might conflicts with the delayed HLE (not sure whether HTTPClient can be used by more than 1 PSP thread at the same time or not tho) |
Thats plausible, the game seems to be doing networking things from multiple threads (theres the |
|
I implemented This leaves my working tree to get LBP PSP connecting as such: All this combined makes the initial login process 100% consistent (eg. im able to get to the community moon) |
|
May i know where you get the definition for |
Someone just told me the constants over discord, and i realized that the constants they sent me did not match what was in the source code as is, so i assumed the ones they sent me were correct, since they also gave a detailed description about the method in question (seems to be worded like it was pulled from some docs somewhere?) and seemed knowledgable, and the game seems to react better when i use the updated constants |
… along with sceHttp syscalls
|
Looks like |
|
It would be great if we could achieve cross platform discovery of room lists online that supports both the IPv6 server and client. |
|
PSP only support/know IPv4 if i'm not mistaken. |
It would be great if the simulator could have a built-in function of creating rooms and adding rooms. Both mobile phones and computers can be online. At present, no teammates can be seen in Chinese Mainland, so it can only be realized by third-party software. We have created an online tool for online use. |
|
|
If you're talking about the ProOnline of AdHoc multiplayer, it's out of the scope of sceHttp library (which is part of Infrastructure networking and unrelated to AdHoc networking) |
This is what i was working on before Mega Man Powered Up official server shutdown in November 2022.
This code is still experimental (just to get things to work), but my implementation is kinda wrong compared to JPCSP implementation.
My implementation is to use the parent simply as a template/base to initialize it's contents (ie. headers, userAgent, etc), thus any changes (ie. cookies, userAgent, etc) to the parent won't reflects existing children.
But later, i find out that JPCSP implementation is id-based relationship between parent-child similar to database's records relationship) and changes on the parent will reflects on it's children, thus cookies can be shared on multiple children (like a multi-tabs on most browsers).
So, this PR will need to be rewritten (probably using naett to support HTTPS too in the future), also need to do communication in a background thread to avoid stuttering the emulation.
So leaving this as a draft until i had the time to rewrite it (in case i lost my local copy due to bad sector).
Tested to work on:
Yu-Gi-Oh! DX Tag Force 2 &Monster Hunter games - partially fixes[Feature Request] Support in-game web browser #14637