2251-2300 of 10000 results (46ms)
2018-01-08 §
12:01 <moritzm> rebooting mw1221-mw1235 for kernel security update (along with update to HHVM 3.18.6 where applicable) [production]
11:38 <moritzm> rebooting mwdebug* for kernel security update [production]
11:28 <godog> puppet node deactivate wtp10[568] - T177374 [production]
11:06 <jdrewniak@tin> Synchronized portals: Wikimedia Portals Update: [[gerrit:402805|Bumping portals to master (T128546)]] (duration: 00m 51s) [production]
11:05 <jdrewniak@tin> Synchronized portals/prod/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:402805|Bumping portals to master (T128546)]] (duration: 00m 51s) [production]
10:50 <godog> roll restart swift in codfw for kernel upgrades [production]
10:40 <akosiaris@tin> Finished deploy [servermon/servermon@53b81d8]: Update servermon (duration: 00m 02s) [production]
10:40 <akosiaris@tin> Started deploy [servermon/servermon@53b81d8]: Update servermon [production]
10:32 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1067 and db1089 - T162807 (duration: 00m 50s) [production]
10:26 <hashar> Started docker on contint1001 / contint2001 . They were missing the overlay/overlayfs kernel modules | T184410 [production]
10:04 <elukey> drain + reboot analytics1029,1031->1034 for kernel updates [production]
10:03 <jynus> fixing wrong events on db2039, db1071,db2023, db2045, db2052, db1100 [production]
09:53 <godog> Flashing Smart Array P840 in Slot 3 [ 4.52 -> 6.06 ] on ms-be2037 - T184390 T141756 [production]
09:46 <hashar> rebooting CI [production]
09:46 <godog> reboot ms-be2037 - T184390 [production]
09:39 <elukey> set sysctl -w net.netfilter.nf_conntrack_tcp_timeout_time_wait=65 to mw1261,mw2251,mw1276 and all videoscalers (Recently rebooted/reimaged) [production]
09:38 <hashar> upgrading contint1001 / contint1002 | T184267 [production]
09:24 <ema> cache_misc: upgrade to latest jessie point release (8.10) T182656 and linux kernel 4.9.65-3+deb9u1~bpo8+2 (KPTI) T184267 [production]
09:17 <_joe_> starting 3 manual loops for consuming refreshLinks jobs for ruwiki [production]
09:14 <marostegui> Force BBU relearn on db1059 - T184160 [production]
08:30 <moritzm> installing remaining openssl updates [production]
07:24 <marostegui> Stop MySQL on db1039 for decommission - T184262 [production]
07:17 <marostegui> Remove db1039 from tendril - T184262 [production]
07:05 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1039 as it will be decommissioned - T184262 (duration: 00m 50s) [production]
07:04 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1039 as it will be decommissioned - T184262 (duration: 00m 50s) [production]
06:51 <marostegui> Stop replication in sync on db1067 and db1089 - T162807 [production]
06:45 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1067 and db1089 - T162807 (duration: 00m 51s) [production]
06:32 <marostegui> Disable auto-learn on db1011 [production]
06:17 <marostegui> Deploy schema change on s7 primary master (db1062) - T174569 [production]
02:33 <l10nupdate@tin> scap sync-l10n completed (1.31.0-wmf.15) (duration: 06m 17s) [production]
2018-01-07 §
20:25 <demon@tin> Synchronized wmf-config/interwiki.php: auto-sync with my plugin was busted 🙃 (duration: 00m 50s) [production]
19:56 <demon@tin> Synchronized php-1.31.0-wmf.15/maintenance/Maintenance.php: fix stuff (duration: 00m 51s) [production]
19:32 <demon@tin> Finished scap: Delete alswik(ibooks|iquote|tionary), mowik(ipedia|tionary) (duration: 21m 32s) [production]
19:10 <demon@tin> Started scap: Delete alswik(ibooks|iquote|tionary), mowik(ipedia|tionary) [production]
08:52 <elukey> re-enabled puppet on db110[78] - eventlogging_sync restarted on db1108 (analytics-slave) - T168414 [production]
2018-01-06 §
08:09 <elukey> re-enable eventlogging mysql consumers after database maintenance - T168414 [production]
06:59 <elukey> set sysctl -w net.netfilter.nf_conntrack_tcp_timeout_time_wait=65 on mw[1329-1333] (new appservers, was 120) [production]
06:49 <elukey> set sysctl -w net.netfilter.nf_conntrack_tcp_timeout_time_wait=65 on mw1335 (new jobrunner, was 120) [production]
2018-01-05 §
22:27 <tgr> T184263 ran mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=eswiki --logwiki=metawiki "Mega849" "Mega809" [production]
20:47 <demon@tin> Pruned MediaWiki: 1.31.0-wmf.12 [keeping static files] (duration: 02m 11s) [production]
18:15 <otto@tin> Finished deploy [analytics/superset/deploy@990bc38]: Running superset with python3 (fingers crossed) (duration: 00m 19s) [production]
18:15 <otto@tin> Started deploy [analytics/superset/deploy@990bc38]: Running superset with python3 (fingers crossed) [production]
18:14 <otto@tin> Finished deploy [analytics/superset/deploy@990bc38]: Running superset with python3 (fingers crossed) (duration: 02m 11s) [production]
18:11 <otto@tin> Started deploy [analytics/superset/deploy@990bc38]: Running superset with python3 (fingers crossed) [production]
16:40 <jynus> upgrade and restart labsdb1010 [production]
16:29 <akosiaris@tin> Finished deploy [servermon/servermon@cf88f3f]: Update servermon to 3c8538a (duration: 00m 02s) [production]
16:29 <akosiaris@tin> Started deploy [servermon/servermon@cf88f3f]: Update servermon to 3c8538a [production]
16:07 <akosiaris@tin> Finished deploy [servermon/servermon@3c8538a]: Update servermon to 3c8538a (duration: 00m 02s) [production]
16:07 <akosiaris@tin> Started deploy [servermon/servermon@3c8538a]: Update servermon to 3c8538a [production]
16:06 <akosiaris@tin> Started deploy [servermon/servermon@3c8538a]: Update servermon to 3c8538a [production]