Рейтинг:0

How to figure out what is bad about a 400 bad request, on an Apache-server

флаг ar

The overarching question

How do I see what is 'bad' about a 400 - bad request?


Info about the error

  • When I click around the WordPress-backend, then between 3 and 7 requests (out of 95-100) give me a: 400 - Bad request-error, upon every page load. But it varies which request that fails, from refresh to refresh?!
  • I've been battling this issue for 8 hours (ish). I feel like I've tried everything.
  • It happens both in Chrome and Firefox
  • It doesn't happen on another computer with an identical setup.
  • It's mainly static files, where the requests fail (but not exclusively).
  • There are 10-15 other developers also working on it, they haven't seen it either (same branch, almost identical database, same Vagrant setup).

System info

  • WordPress-installation
  • Apache-server
  • Running locally on a Vagrant box (Ubuntu VirtualBox)

Solution attempt 1: See tendencies between which files it fails to load

I can't see any tendencies there. Here are 4 random refreshes.

Refresh1

api-client-1740.js
backbone.min.js
styleGuide-1740.js
escape-html.js
admin-script.min.js
quicktags.js
admin-bar.js
hooks.js
debug-bar-js.dev.js
font-awesome.css
nav-menus.css
list-tables.css

Refresh2

nav-menus.css
about.css
admin-menu.css

Refresh3

tags-suggest.js
i18n.js
wp-polyfill.js
themes.css
list-tables.css

Refresh4

"Image loadingAnimation.gif"
flame.png
api-client-1740.js
element.min.js
admin-script.min.js
moment.min.js
mouse.min.js
lodash.min.js
underscore.min.js
hoverintent-js.min.js
adminbar-1740.css
admin-global-1740.css

And sometimes none of the requests fails.


Solution attempt 2: Inspect the request in the network-tab

I can find a reason for the 400 Bad request in there. Here is an example of a request that returned 400 Bad request (where I've redacted sensitive info).

Request

GET /wp/wp-admin/load-styles.php?c=0&dir=ltr&load%5Bchunk_0%5D=dashicons,admin-bar,site-health,common,forms,admin-menu,dashboard,list-tables,edit,revisions,media,themes,about,nav-menus,wp-poi&load%5Bchunk_1%5D=nter,widgets,site-icon,l10n,buttons,wp-auth-check&ver=5.8.3 HTTP/1.1
Host: mydomain.test
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0
Accept: text/css,*/*;q=0.1
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://mydomain.test/wp/wp-admin/
DNT: 1
Connection: keep-alive
Cookie: wordpress_a8d2d12312312379974dc3f916b356d3=MYWORDPRESSUSERNAME%7C1612378456%7C8xaOuvvv7N7Aq4PzHreiIfjKNoRt6NInOxbwvWW9nYL%7Cc70240d7f46d0dc5b6112554da95f69cee61180ebb2b3a93326cb42d02937f3f; wordpress_test_cookie=WP%20Cookie%20check; wordpress_logged_in_a8d2dfda29fb0e79974dc3f916b356d3=MYWORDPRESSUSERNAME%7C1612378456%7C8xaOuvvv7N7Aq4PzHreiIfjKNoRt6NInOxbwvWW9nYL%7Cd16311d6028a3550a8777754a99a1e2d09bffb05f45f36854038387bc0fdce43; wp-settings-2=posts_list_mode%3Dlist%26ampampampamplibraryContent%3Dbrowse%26ampampampampeditor%3Dtinymce%26ampampampampmfold%3Do%26libraryContent%3Dbrowse; wp-settings-time-2=1652706157; CookieConsent={stamp:%27bs7eydGl1MNS0O2BJZiPnLkz4NSuHl+9jWJceT1L8gnBXxa3B+MlsA==%27%2Cnecessary:true%2Cpreferences:true%2Cstatistics:true%2Cmarketing:true%2Cver:1%2Cutc:1652701927276%2Ciab2:%27CPZD8gAPZD8gACGABBENCPCsAP_AAH_AAAAAIwtd_X__bX9j-_5_bft0eY1P9_r3_-QzjhfNs-8F3L_W_L0Xw2E7NF36pq4KuR4Eu3LBIQNlHMHUTUmwaokVrzHsak2cpyNKJ7LEknMZO2dYGH9Pn9lDuYKY7_5___bx3D-v_t_-39T378Xf3_d5_2_--vCfV599jbn9fV_7_9nP___9v-_8_________gjAASYal5AF2JY4MmkaRQogRhWEhVAoAKKAYWiKwAcHBTsrAJdQQsAEAqAjAiBBiCjBgEAAgEASERASAFggEQBEAgABAAiAQgAImAQWAFgYBAAKAaFiAFAAIEhBkQERymBARIlFBLZWIJQV7GmEAdZYAUCiMioAESAAAkBASFg5jgCQEuFkgSYoXyAEYIUAAAAA.YAAAAAAAAAAA%27%2Cgacm:%271~AAAAAAAAAAACAABAAAgAIAAABAAhAAAACAAAAAAAQAAQAAAAAAABBBAAIAAAAAAAAAAAAQAAAIBAAAAAIgMAAAAAAAgAAAACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAgAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAQAAAAAAFAAAABAAAAAAAAAAAARBAAAAAAAAAACAAABAAAAAAAAAAEAAAAAAABAAAAAEAAAAAAAAAAAAAAAAAAACBAAAAAAAAAAAAQAAAAAAAAAAgAAAAAAAQAAAAAAAAAAAAAAAACAgBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAgAAAAAAAEAAAAAAAQAAgAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAkQAAAAAAAAAAAAAAAAQ=%27%2Cregion:%27dk%27}; modal_page_counter_cookie=1; utag_main=v_id:0180ccb62e2b002302377014532c05054001000f00fbe$_fd:2$_se:1$_ss:1$_st:1652709110390$dc_visit:2$ses_id:1652707310390%3Bexp-session$_pn:1%3Bexp-session$dc_event:1%3Bexp-session; user_obj=eyJ0b2tlbiI6ImQ1YWFhYWI5NzQyZTc2M2FlZjEyYzhjNDBhNGE3NWRhODA3MmZhOTc4MDNkZDdhMWJkNmRlNzYzODRhMjM4MDgiLCJzdWJzY3JpcHRpb25fYXJlYXMiOnsiMCI6IjlkYjk2OTcxLTFiNmItNDM0NC05OWFiLTg5YTVjM2NkYmU5MSIsIjNhMGYxODU3LWMzOTMtNGI4Ny1hNjIwLTE4YzEyMTA2OWMdfiI6IjNhMGYxODU3LWMzOTMtNGI4Ny1hNjIwLTE4YzEyMTA2OWM3ZiIsIjNhMDg2OGEzLThiZWEtNGE1Mi1hMGUwLTI4OWE0ZTNkMzIxMSI6IjNhMDg2OGEzLThiZWEtNGE1Mi1hMGUwLTI4OWE0ZTNkMzIxMSIsIjlkYjk2OTcxLTFiNmItNDM0NC05OWFiLTg5YTVjM2NkYmU5MSI6IjlkYjk2OTcxLTFiNmItNDM0NC05OWFiLTg5YTVjM2NkYmU5MSIsIjE0ZmU1ZDI0LWQxYTYtNGNmNy04NmNhLTgzZTlhNDgzNDkIJGkdf4jE0ZmU1ZDI0LWQxYTYtNGNmNy04NmNhLTgzZTlhNDgzNDk5OCIsIjcwZjI1NjRjLTI2ZTItNDU5Yy05MmQ2LTMwOTUwMWQ4MTc1OSI6IjcwZjI1NjRjLTI2ZTItNDU5Yy05MmQ2LTMwOTUwMWQ4MTc1OSIsIjc3ZDgzMGZlLTYwNzYtNDRjMy05MTZmLTMwNzUwODNkZDg1MiI6Ijc3ZDgzMGZlLTYwNzYtNDRjMy05MTZmLTMwNzUwODNkZDg1MiIsIjMxNjNiY2RhLTIxYWEtNDhlZS1hYmFlLTI2NjAzZTA3YjU4NSI6IjMxNjNiY2RhLT5YWEtNDhlZS1hYmFlLTI2NjAzZTA3YjU4NSIsIjQwMzcyMzEzLTZkYTItNDJhZi04NzBlLWRmMGE5MDU2MmZmNiI6IjQwMzcyMzEzLTZkYTItNDJhZi04NzBlLWRmMGE5MDU2MmZmNiIsImE3ZGQzMjc2LTJiNDMtNDg2NS1iNGViLTFiZTEwMDk3NmQ5NiI6ImE3ZGQzMjc2LTJiNDMtNDg2NS1iNGViLTFiZTEwMDk3NmQ5NiIsImUxY2ExNzc2LTU0NTUtNDc4MC04OTMzLTgxM2M0NjlmZmZhMSI6ImUxY2ExNzc2LTU0NTUtNDc4MC071TMzLTgxM2M0NjlmZmZhMSIsImY2ZWQ5NmVhLWU4MjAtNDg3My1iMmM2LTcwYzRjODc1ZjUyMCI6ImY2ZWQ5NmVhLWU4MjAtNDg3My1iMmM2LTcwYzRjODc1ZjUyMCIsImY3YjViYjRjLTZmYWUtNDdhYi05YTgzLWUyNGVjMzllZWY1ZSI6ImY3YjViYjRjLTZmYWUtNDdhYi05YTgzLWUyNGVjMzllZWY1ZSJ9LCJlbWFpbCI6InpvZCtzdWJzY3JpcHRpb250eXBlMkBwZXl0ei5kayIsInN1YnNjcmlwdGlvbl90eXBlIjoiMiJ9; user_token=d5aaaab9742e763aef12c8c40a4a75da8072fa97803dd7a1bd6de76384a23808; wordpress_test_cookie=WP%20Cookie%20check; wordpress_logged_in_a8d2d12312312379974dc3f916b356d3=MYWORDPRESSUSERNAME%7C1612378456%7C8xaOuvvv7N7Aq4PzHreiIfjKNoRt6NInOxbwvWW9nYL%7Cc70240d7f46d0dc5b6112554da95f69cee61180ebb2b3a93326cb42d02937f3f
Pragma: no-cache
Cache-Control: no-cache

Response headers

HTTP/1.1 400 Bad Request
Date: Mon, 16 May 2022 13:27:53 GMT
Server: Apache/2.4.41 (Ubuntu)
Content-Length: 299
Connection: close
Content-Type: text/html; charset=iso-8859-1

Response

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
<hr>
<address>Apache/2.4.41 (Ubuntu) Server at vagrant Port 80</address>
</body></html>

I can't see out of that, why that request was 'bad'. Nor why the server couldn't understand it.


Solution attempt 3: Check the logs

I checked these:

/var/log/apache2/access.log
/var/log/apache2/error.log
/var/log/apache2/other_vhosts_access.log

I even tried only outputting lines containing the word 400, with this command:

tail -f access.log error.log other_vhosts_access.log | awk '/400/'

And I could get them (from the other_vhost_access.log-file). But I can't see any reason for it to be 'bad' either:

In Firefox

vagrant:80 123.123.123.1 - - [16/May/2022:15:27:53 +0200] "GET /wp/wp-admin/js/common.min.js HTTP/1.1" 400 481 "http://mydomain.test/wp/wp-admin/, http://mydomain.test/wp/wp-admin/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0, Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0"

In Chrome

vagrant:80 123.123.123.1 - - [16/May/2022:15:45:24 +0200] "POST /es/query/ HTTP/1.1" 400 481 "http://mydomain.test/, http://mydomain.test/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36, Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36"

Solution attempt 4: Change LogLevel on the Apache-server

As advised here I tried Changing the LogLevel on the Apache-server.

But I didn't see any better logs in access.log, error.log nor other_vhost_access.log.


Solution attempt 5: Debug using tcpdump

I also tried running tcpdump from inside the box and output it all using the command:

13:01:42.102438 IP (tos 0x0, ttl 64, id 54239, offset 0, flags [DF], proto TCP (6), length 52)
    123.123.123.12.80 > 123.123.123.4.57340: Flags [.], cksum 0xc3db (incorrect -> 0xe5ed), seq 114948251, ack 2535382125, win 505, options [nop,nop,TS val 243067183 ecr 1641029207], length 0
13:01:42.102475 IP (tos 0x0, ttl 64, id 8630, offset 0, flags [DF], proto TCP (6), length 52)
    123.123.123.12.80 > 123.123.123.4.57334: Flags [.], cksum 0xc3db (incorrect -> 0xbf8a), seq 2046057008, ack 256413570, win 505, options [nop,nop,TS val 243067183 ecr 3257787508], length 0
13:01:42.106114 IP (tos 0x2,ECT(0), ttl 64, id 54240, offset 0, flags [DF], proto TCP (6), length 533)
    123.123.123.12.80 > 123.123.123.4.57340: Flags [P.], cksum 0xc5bc (incorrect -> 0xe068), seq 114948251:114948732, ack 2535382125, win 505, options [nop,nop,TS val 243067186 ecr 1641029207], length 481: HTTP, length: 481
    HTTP/1.1 400 Bad Request
    Date: Mon, 16 May 2022 11:01:42 GMT
    Server: Apache/2.4.41 (Ubuntu)
    Content-Length: 299
    Connection: close
    Content-Type: text/html; charset=iso-8859-1

    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html><head>
    <title>400 Bad Request</title>
    </head><body>
    <h1>Bad Request</h1>
    <p>Your browser sent a request that this server could not understand.<br />
    </p>
    <hr>
    <address>Apache/2.4.41 (Ubuntu) Server at vagrant Port 80</address>
    </body></html>
13:01:42.106394 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)
    123.123.123.4.57340 > 123.123.123.12.80: Flags [.], cksum 0xddfb (correct), seq 2535382125, ack 114948732, win 2051, options [nop,nop,TS val 1641029211 ecr 243067186], length 0
13:01:42.109623 IP (tos 0x0, ttl 64, id 54241, offset 0, flags [DF], proto TCP (6), length 52)
    123.123.123.12.80 > 123.123.123.4.57340: Flags [F.], cksum 0xc3db (incorrect -> 0xe400), seq 114948732, ack 2535382125, win 505, options [nop,nop,TS val 243067190 ecr 1641029211], length 0

And again, I can see the 400 Bad request-error. But I can't see what's bad about it.


Solution attempt 6: Look into headers / cookies

Since the errors only occurred when I was logged in, then I tried looking at the header and cookies. But I couldn't see anything spectacular there.

I even compared them to the headers/cookies on another machine, where this is running flawlessly, and nothing sprung in my eye.

... And still... Even if it was this, then why don't I get a better error message?


Solution attempt 7: Server performance - activity monitor

I checked the disk-space, CPU-usage and memory usage on the server (via bpytop) when I refreshed a page. But it looks normal.


Untried attempts

  • Downgrade and reinstall and/or upgrade WordPress core.
  • Stop and start apache
  • Restart the Ubuntu-server.

I've not done this, since I would really like to find out, how to see what is wrong here (instead of just trying to fix it).

флаг us
Rob
Возможно, запись виртуального хоста Apache указывает (ошибки) файлы журнала в другом месте, или вы настроили что-то вроде: https://serverfault.com/q/997624/960939, потому что, насколько мне известно, в противном случае должно быть зарегистрировано 400 ошибок. Когда вы используете WordPress, очень вероятно, что у вас активны правила перезаписи, возможно, они работают неправильно — https://httpd.apache.org/docs/2.4/mod/mod_rewrite.html#logging
djdomi avatar
флаг za
сайт для личных целей?
Tilman Schmidt avatar
флаг bd
Просмотрите полный нефильтрованный файл error.log на момент возникновения ошибки. Не фильтруйте по «400» — нет причин, по которым сообщение о том, что происходит не так, должно содержать эту строку.
Nikita Kipriyanov avatar
флаг za
Не разбивайте свои усилия на «попытки». Вы должны учитывать все факторы одновременно: журнал доступа, журнал ошибок, сеть (фактические «байты» запроса и ответа), представление клиента. Вы должны изолировать записи всех из них, которые соответствуют одному и тому же единственному неправильному запросу. Кроме того, вы можете настроить Apache для более подробного ведения журнала. Кроме того, вы видели сам «плохой запрос» — повторите этот запрос с полными заголовками, используя curl или даже «telnet webserver 80» или openssl s_client (в случае HTTPS), убедитесь сами, действительно ли это плохой запрос или временный. проблема на стороне веб-сервера.

Ответить или комментировать

Большинство людей не понимают, что склонность к познанию нового открывает путь к обучению и улучшает межличностные связи. В исследованиях Элисон, например, хотя люди могли точно вспомнить, сколько вопросов было задано в их разговорах, они не чувствовали интуитивно связи между вопросами и симпатиями. В четырех исследованиях, в которых участники сами участвовали в разговорах или читали стенограммы чужих разговоров, люди, как правило, не осознавали, что задаваемый вопрос повлияет — или повлиял — на уровень дружбы между собеседниками.