[Bug 57151] New: AppDB is incredibly slow
https://bugs.winehq.org/show_bug.cgi?id=57151 Bug ID: 57151 Summary: AppDB is incredibly slow Product: WineHQ Apps Database Version: unspecified Hardware: x86-64 OS: Linux Status: NEW Keywords: source Severity: normal Priority: P2 Component: appdb-unknown Assignee: wine-bugs(a)winehq.org Reporter: imwellcushtymelike(a)gmail.com Distribution: --- The AppDB seems to be getting slower and slower. Today it's taking up to a minute just to load a page. Does it need more resources? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 Ken Sharp <imwellcushtymelike(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|normal |major --- Comment #1 from Ken Sharp <imwellcushtymelike(a)gmail.com> --- Still getting worse. Cloudflare responds that the server is not responding more often than it actually works. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 Jeremy Newman <jnewman(a)codeweavers.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jnewman(a)codeweavers.com --- Comment #2 from Jeremy Newman <jnewman(a)codeweavers.com> --- Bots are hammering the site. We will need to do a bit more bot mitigation. I don't want to bump up the specs of the VPS unless we absolutely have to. We could perhaps consider locking down more of the site behind a login and preventing bots from spidering things. Disable anonymous browsing of the content. The browse links and app view pages could require a login. Depends on how much we really care if our search results for the AppDB appear in Google. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 --- Comment #3 from Ken Sharp <imwellcushtymelike(a)gmail.com> --- Are these "bad" bots? Would they respond to being told to slow down (HTTP 429)? Of course then you're playing whack-a-bot. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 F H <francisco278herrera(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |francisco278herrera(a)gmail.c | |om --- Comment #4 from F H <francisco278herrera(a)gmail.com> --- (In reply to Jeremy Newman from comment #2)
Bots are hammering the site. We will need to do a bit more bot mitigation. I don't want to bump up the specs of the VPS unless we absolutely have to.
We could perhaps consider locking down more of the site behind a login and preventing bots from spidering things. Disable anonymous browsing of the content. The browse links and app view pages could require a login. Depends on how much we really care if our search results for the AppDB appear in Google.
what about implementing captchas? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 --- Comment #5 from F H <francisco278herrera(a)gmail.com> --- I believe captchas to be a better solution than locking parts of the site behind logins because they reduce friction and the appdb entries could still appear on google. it could have an automated captcha, a checkbox captcha or a image recognition captcha depending on how likely the user is to be a bot and website load, cloudflare offers these captchas -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 --- Comment #6 from Ken Sharp <imwellcushtymelike(a)gmail.com> --- *** Bug 57731 has been marked as a duplicate of this bug. *** -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 --- Comment #7 from Ken Sharp <imwellcushtymelike(a)gmail.com> --- *** Bug 57730 has been marked as a duplicate of this bug. *** -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 --- Comment #8 from Ken Sharp <imwellcushtymelike(a)gmail.com> --- *** Bug 57732 has been marked as a duplicate of this bug. *** -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 Ken Sharp <imwellcushtymelike(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |56705, 56706 -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 Jeremy Newman <jnewman(a)codeweavers.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO --- Comment #9 from Jeremy Newman <jnewman(a)codeweavers.com> --- Since we are using CloudFlare to proxy that site, I've enabled the "I'm Under Attack" security level which enforces a captcha to browse the site. That immediately dropped the load averages from over 7.0 down in the 1.0 range. The site should be usable now. While still slow, we should see less complete unable to load situations.
From this point on the focus will need to be on optimization. I have bugs about creating table indexes, and splitting the appdb and bugs mysql server connections. Both of those should help out loads as well. Hopefully someone steps up and tackles those.
What we won't be doing is increasing the size of the VPS, that is a band-aid and a waste of our funds. There should be no reason that this site cannot run well on a 8GB RAM 4 core VPS. We can keep this bug open as a meta bug, since the site is still slow until we solve those other bugs (and probably more). -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 --- Comment #10 from F H <francisco278herrera(a)gmail.com> --- Even with the captcha on I still get this error when trying to see queued items on the AppDB Ooops! Something has gone terribly wrong! Our monkey train has derailed! Worry not, a webmaster gopher help army has been dispatched and is on the way. If this error continues to be a problem, please report it to us on our Forums error details: Error Message: Uncaught Error: Non-static method Bug::listSubmittedBy() cannot be called statically in /home/winehq/opt/appdb/queueditems.php:50 Stack trace: #0 {main} thrown File: queueditems.php:50 Privacy Policy If you have a privacy inquiry regarding this site, please write to privacy(a)winehq.org -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 --- Comment #11 from F H <francisco278herrera(a)gmail.com> --- (In reply to F H from comment #10)
Even with the captcha on I still get this error when trying to see queued items on the AppDB
Ooops! Something has gone terribly wrong! Our monkey train has derailed! Worry not, a webmaster gopher help army has been dispatched and is on the way.
If this error continues to be a problem, please report it to us on our Forums
error details: Error Message: Uncaught Error: Non-static method Bug::listSubmittedBy() cannot be called statically in /home/winehq/opt/appdb/queueditems.php:50 Stack trace: #0 {main} thrown
File: queueditems.php:50
Privacy Policy If you have a privacy inquiry regarding this site, please write to privacy(a)winehq.org
While logged in -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 --- Comment #12 from Ken Sharp <imwellcushtymelike(a)gmail.com> --- Indeed there is an error occurring when trying to process submitted items. error details: Error Message: Uncaught Error: Undefined constant "iTestingId" in /home/winehq/opt/appdb/include/testData.php:972 Stack trace: #0 /home/winehq/opt/appdb/include/testData_queue.php(90): testData->outputEditor() #1 /home/winehq/opt/appdb/include/version_queue.php(198): testData_queue->outputEditor() #2 /home/winehq/opt/appdb/include/objectManager.php(405): version_queue->outputEditor() #3 /home/winehq/opt/appdb/objectManager.php(137): ObjectManager->display_entry_for_editing() #4 {main} thrown File: testData.php:972 -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 --- Comment #13 from Jeremy Newman <jnewman(a)codeweavers.com> --- These are not related to speed. These recent fatal errors are due to the upgrade to PHP8 which is more strict than 7 was. AppDB MR!2 solved most of the PHP8 issues, but not surprised there are more knowing how big of a change 8 is. Please use bug 57734 to track those fatal errors. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 Ken Sharp <imwellcushtymelike(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |57734 -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 Chiitoo <chiitoo(a)gentoo.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |chiitoo(a)gentoo.org -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 Tom Maneiro <tomman(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomman(a)gmail.com --- Comment #14 from Tom Maneiro <tomman(a)gmail.com> --- Unfortunately, enabling Cloudflare in "I'm under attack" level effectively bars those of us not using mainstream web browsers[1] (I'm using SeaMonkey, but this also impacts a bunch of small/niche/developing projects) because of failures to get past Turnstile (the captcha/checkbox thing). As of February 5, I don't even get a clickable checkbox anymore that would eventually let me pass, instead all I get is a endless page reload loop with the "Verify if you're human" CF headers. I know this is out of scope for Wine to fix (other than dropping down CF's protection a few notches, defeating the purpose of it), but this effectively locks me out from even accessing the AppDB, much less maintaining my applications. Using another web browser is not an acceptable option in any case, unfortunately. So I guess it's matter of sitting and wait while the storm goes away... [1] https://news.ycombinator.com/item?id=42953508 -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 Bug 57151 depends on bug 56705, which changed state. Bug 56705 Summary: AppDB tables need indexes https://bugs.winehq.org/show_bug.cgi?id=56705 What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |FIXED -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 --- Comment #15 from Tom Maneiro <tomman(a)gmail.com> --- To whoever removed / disabled / relaxed Cloudflare protection on the AppDB: thanks~! (I'm aware that this must not have been a easy decision, but for the greater good of FOSS in these complex times, it's welcome nonetheless) I can now use the AppDB again from my web browser of choice, so far some minor wait times but no big deal - better than being completely locked out by a uninterested 3rd party! -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 --- Comment #16 from Jeremy Newman <jnewman(a)codeweavers.com> --- I did some work adding some indexes to our DB tables. This got the load under control enough I felt the site was "usable" now without the "I'm Under Attack" setting. If the load average jumps back up over 5.0 for extended periods I will re-enable it temporarily. I'm also looking into other solutions to handling the bot problem. mod_evasive is a helpful one to setup. A more nuclear option would be to enable this: https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=57151 Bug 57151 depends on bug 57734, which changed state. Bug 57734 Summary: PHP8 Fatal Errrors https://bugs.winehq.org/show_bug.cgi?id=57734 What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |RESOLVED Resolution|--- |FIXED -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=57151 Bug 57151 depends on bug 56706, which changed state. Bug 56706 Summary: AppDB does not properly separate the Bugzilla DB http://bugs.winehq.org/show_bug.cgi?id=56706 What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |RESOLVED Resolution|--- |FIXED -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
participants (1)
-
WineHQ Bugzilla