|
2021-06-26
§
|
| 21:23 |
<volans> |
uploaded spicerack_0.0.56 to apt.wikimedia.org buster-wikimedia,bullseye-wikimedia |
[production] |
| 16:37 |
<elukey> |
restart php-fpm on mw1387 |
[production] |
| 15:53 |
<Jayprakash12345> |
Deploying T285598 and T285599 |
[tools.bodh] |
| 15:52 |
<Jayprakash12345> |
Deploying T285596 |
[tools.bodh-backend] |
| 15:43 |
<elukey> |
restart php-fpm on mw1393 |
[production] |
| 15:39 |
<elukey> |
restart php-fpm on mw1405 mw1399 mw1385 |
[production] |
| 15:37 |
<elukey> |
restart php-fpm on mw1397 mw1395 mw1411 mw1407 |
[production] |
| 15:31 |
<elukey> |
restart php-fpm on mw1391 mw1389 mw1403 |
[production] |
| 13:49 |
<elukey> |
restart php-fpm on mw1368 mw1370 mw1366 mw1409 |
[production] |
| 13:43 |
<elukey> |
depool mw1384 for investigation |
[production] |
| 13:43 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=mw1384.eqiad.wmnet |
[production] |
| 13:33 |
<elukey> |
restart phpfpm on mw1353 mw1365 mw1371 |
[production] |
| 13:30 |
<elukey> |
restart php-fpm on mw1351 mw1373 mw1352 mw1349 |
[production] |
| 13:23 |
<elukey> |
restart-phpfpm on mw1350 (0 idle php workers) |
[production] |
| 13:20 |
<elukey> |
restart-phpfpm on mw1333 (0 idle php workers) |
[production] |
| 10:08 |
<elukey> |
restart php-fpm on mw1372 - T285593 |
[production] |
| 10:07 |
<elukey> |
restart php-fpm on mw1372 - T285593 |
[production] |
| 09:45 |
<elukey> |
restart php-fpm on mw135[4-5] |
[production] |
| 09:44 |
<elukey> |
restart php-fpm on mw1354 |
[production] |
| 09:38 |
<elukey> |
reboot mw1414 (not reachable via ssh, nor via mgmt console) |
[production] |
| 09:33 |
<elukey> |
restart php-fpm on mw1367 (php fatal memory errors, php7adm /apcu-frag returns errors) |
[production] |
|
2021-06-25
§
|
| 21:50 |
<wikibugs> |
Updated channels.yaml to: 45c892ec66874f389a71e57dae009c15430079fd Remove #wikimedia-ve, Wikimedia-Venezuela project was archived |
[tools.wikibugs] |
| 21:37 |
<ebernhardson@deploy1002> |
Synchronized php-1.37.0-wmf.11/extensions/CirrusSearch/: cirrus: Revert "Stop querying ores_articletopic" (3/3) (duration: 01m 01s) |
[production] |
| 21:35 |
<ebernhardson@deploy1002> |
Synchronized php-1.37.0-wmf.11/extensions/CirrusSearch/includes/Wikimedia/WeightedTagsHooks.php: cirrus: Revert "Stop querying ores_articletopic" (2/3) (duration: 00m 58s) |
[production] |
| 21:34 |
<ebernhardson@deploy1002> |
Synchronized php-1.37.0-wmf.11/extensions/CirrusSearch/includes/Parser/FullTextKeywordRegistry.php: cirrus: Revert "Stop querying ores_articletopic" (1/3) (duration: 00m 58s) |
[production] |
| 20:32 |
<legoktm> |
legoktm@mwmaint1002:~$ sudo systemctl reset-failed # to clear icinga alert |
[production] |
| 20:28 |
<legoktm> |
legoktm@mwmaint1002:~$ sudo systemctl start mediawiki_job_update_special_pages.service (T285583) |
[production] |
| 20:21 |
<urbanecm@deploy1002> |
Synchronized php-1.37.0-wmf.11/extensions/GrowthExperiments/modules/homepage/suggestededits/ext.growthExperiments.Homepage.SuggestedEdits.js: eaec745e4504527d23ddca32eb7fcd531d5553f9: SuggestedEdits: Only log task impression for EditCardWidget (T283546; emergency deployment) (duration: 01m 00s) |
[production] |
| 18:08 |
<legoktm> |
legoktm@ms-fe2005:~$ sudo systemctl unmask swiftrepl-mw.service |
[production] |
| 15:46 |
<mutante> |
mw1326, mw1327, mw1328, mw1329 ... restarted php-fpm |
[production] |
| 15:41 |
<mutante> |
mw1330, mw1320, mw1321, mw1322 - restarted php-fpm |
[production] |
| 15:38 |
<mutante> |
[mw1330:~] $ sudo restart-php7.2-fpm |
[production] |
| 15:36 |
<mutante> |
[mw1332:~] $ sudo restart-php7.2-fpm |
[production] |
| 15:28 |
<mutante> |
[mw1319:~] $ sudo restart-php7.2-fpm |
[production] |
| 15:28 |
<andrewbogott> |
restarting openstack services on cloudcontrol1005 |
[admin] |
| 15:27 |
<wm-bot> |
Adding a new k8s worker node - cookbook ran by dcaro@vulcanus |
[toolsbeta] |
| 15:26 |
<wm-bot> |
Adding a new k8s worker node - cookbook ran by dcaro@vulcanus |
[toolsbeta] |
| 15:21 |
<wm-bot> |
Adding a new k8s worker node - cookbook ran by dcaro@vulcanus |
[toolsbeta] |
| 15:20 |
<rzl> |
rzl@mw1320:~$ sudo restart-php7.2-fpm # workers stuck since the ~14:00 request spike |
[production] |
| 15:19 |
<wm-bot> |
Adding a new k8s worker node - cookbook ran by dcaro@vulcanus |
[toolsbeta] |
| 15:17 |
<wm-bot> |
Adding a new k8s worker node - cookbook ran by dcaro@vulcanus |
[toolsbeta] |
| 15:15 |
<wm-bot> |
Adding a new k8s worker node - cookbook ran by dcaro@vulcanus |
[toolsbeta] |
| 15:08 |
<wm-bot> |
Adding a new k8s worker node - cookbook ran by dcaro@vulcanus |
[toolsbeta] |
| 15:07 |
<wm-bot> |
Adding a new k8s worker node - cookbook ran by dcaro@vulcanus |
[toolsbeta] |
| 15:03 |
<wm-bot> |
Adding a new k8s worker node - cookbook ran by dcaro@vulcanus |
[toolsbeta] |
| 15:03 |
<oblivian@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 15:02 |
<wm-bot> |
Adding a new k8s worker node - cookbook ran by dcaro@vulcanus |
[toolsbeta] |
| 15:00 |
<wm-bot> |
Adding a new k8s worker node - cookbook ran by dcaro@vulcanus |
[toolsbeta] |
| 14:59 |
<wm-bot> |
Adding a new k8s worker node - cookbook ran by dcaro@vulcanus |
[toolsbeta] |
| 14:56 |
<wm-bot> |
Adding a new k8s worker node - cookbook ran by dcaro@vulcanus |
[toolsbeta] |