751-800 of 10000 results (39ms)
2017-07-04 §
08:25 <elukey> restart redis 6380 (slave) jobqueue instance on rdb1004/2003 to force resync with master [production]
08:12 <moritzm> powercycling mw1260, stuck in reboot [production]
07:56 <moritzm> powercycling mw1259, stuck in reboot [production]
07:52 <gehel> restart of relforge for kernel upgrade [production]
07:42 <moritzm> rebooting video scalers in eqiad for kernel update [production]
07:15 <marostegui> Deploy alter table on s3 hosts (eqiad) - T168661 [production]
06:05 <marostegui> Stop MySQL on db1060 for maintenance - T153743 [production]
05:47 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1060 - T153743 (duration: 02m 51s) [production]
05:26 <marostegui> Deploy alter table on s5 directly on s5 master (db1063) - T168661 [production]
05:20 <marostegui> Deploy alter table on s6 directly on s6 master (db1061) - T168661 [production]
05:08 <marostegui> Deploy alter table on s2 directly on s2 master (db1054) - T168661 [production]
02:27 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.7) (duration: 10m 14s) [production]
01:30 <mutante> releases1001: switching GID of reprepro and promemetheus-node-exporter group (1000 vs 1001), changing reprepro UID to 13927. using find -exec to fix all the permissions and make it identical to bromine. prevent permissions snafu when rsyncing (T164030) [production]
2017-07-03 §
20:46 <gehel> unbanning elastic1018 from elasticsearch eqiad cluster [production]
20:24 <gehel> banning elastic1018 from elasticsearch eqiad clsuter [production]
19:29 <hashar> restarting jenkins [production]
19:10 <nuria@tin> Finished deploy [eventlogging/analytics@328dea6]: (no justification provided) (duration: 00m 03s) [production]
19:09 <nuria@tin> Started deploy [eventlogging/analytics@328dea6]: (no justification provided) [production]
17:35 <chasemp> labvirt1003:~# service nova-compute restart [production]
16:55 <bd808> Running maintain-views --all-databases --clean --replace-all --debug on labsdb1001 [production]
16:51 <mobrovac@tin> Finished deploy [mobileapps/deploy@58a5b19]: (no justification provided) (duration: 00m 41s) [production]
16:51 <mobrovac@tin> Started deploy [mobileapps/deploy@58a5b19]: (no justification provided) [production]
16:02 <chasemp> labvirt1002:~# service nova-compute restart [production]
15:43 <mobrovac@tin> Finished deploy [mobileapps/deploy@58a5b19]: Remove pronunciation from the spec - T169299 (duration: 09m 30s) [production]
15:33 <mobrovac@tin> Started deploy [mobileapps/deploy@58a5b19]: Remove pronunciation from the spec - T169299 [production]
15:30 <ema> cp1099: restart varnish-be [production]
15:16 <chasemp> labcontrol1001 clean out admin-monitoring leaks [production]
15:12 <chasemp> labvirt1001 service nova-compute restart [production]
14:40 <elukey> running EventLogging alter tables on dbstore1002 (script in /home/elukey/dbstore1002.sql) - T167162 [production]
14:33 <akosiaris> set enable_notification=0 in icinga [production]
13:54 <moritzm> rebooting ms-be2028 to ms-be2035 for kernel update [production]
13:43 <marostegui> Global rename of Antero de Quintal → JMagalhães - T169527 [production]
13:39 <moritzm> uploaded apache2 2.4.10+deb8u9+wmf1 to apt.wikimedia.org/jessie-wikimedia [production]
12:39 <marostegui> Compress innodb on db2056 - T169510 [production]
12:36 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Add comments about db2056 status - T169510 (duration: 02m 50s) [production]
12:26 <elukey> reimage stat1005 with Debian Stretch [production]
12:03 <moritzm> rebooting scb1004 for kernel update [production]
10:39 <moritzm> rebooting mw1298 for kernel update [production]
09:37 <marostegui> Global rename of Markos90 → Mαρκος - T169396 [production]
09:24 <marostegui> Deploy alter table on s1 directly on codfw master (db2016) and let it replicate - T166204 [production]
09:07 <_joe_> restarting the passenger app on puppetmasters in codfw serially with a sleep of 3 seconds for T169493 [production]
08:58 <_joe_> restarting the passenger app on puppetmaster1002 for T169493 [production]
08:49 <gehel> unbanning elastic1020 from elasticsearch eqiad [production]
08:29 <marostegui> Compress dewiki on dbstore2001 - T168354 [production]
08:25 <gehel> banning elastic1020 from elasticsearch eqiad waiting for its recovery [production]
08:02 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Add comments about db1039 status - T166208 (duration: 02m 49s) [production]
07:51 <marostegui> Deploy alter table db1039 - s7 - T166208 [production]
07:40 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1034 - T166208 (duration: 03m 00s) [production]
07:35 <marostegui> Drop alter table s7 - labsdb1003 - T166208 [production]
07:24 <volans> bounced uwsgi-graphite-web on graphite1003, log stopped since Jul 2 10:23:45 [production]