251-300 of 10000 results (25ms)
2015-07-04 §
12:56 <krinkle> Synchronized php-1.26wmf12/resources/src/mediawiki/mediawiki.Title.js: I1dae1e63e47 (duration: 00m 17s) [production]
05:01 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat Jul 4 05:01:43 UTC 2015 (duration 1m 42s) [production]
03:11 <ori> Promoted Krinkle and Krenair to admin, cloudadmin on wikitech, because duh. [production]
02:39 <LocalisationUpdate> completed (1.26wmf12) at 2015-07-04 02:39:41+00:00 [production]
02:29 <l10nupdate> Synchronized php-1.26wmf12/cache/l10n: (no message) (duration: 09m 59s) [production]
01:00 <springle> reload haproxy dbproxy1004 [production]
2015-07-03 §
23:59 <legoktm> Synchronized php-1.26wmf12/extensions/Translate/: Translate+UserMerge fixes (duration: 00m 17s) [production]
23:55 <legoktm> Synchronized php-1.26wmf12/extensions/WikiLove/: WikiLove+UserMerge fixes (duration: 00m 18s) [production]
23:24 <ori> Synchronized w/404.php: Force 'Transfer-Encoding: Chunked' header on 404 responses (duration: 00m 31s) [production]
22:36 <Krenair> restarted apache on silver to see if it would make https://gerrit.wikimedia.org/r/#/c/221969/ take effect for T104360. It did not. [production]
21:46 <ori> depooled mw1152 [production]
20:12 <ori> restarted cassandra on restbase1001 [production]
17:28 <ori> pooled mw1152 (HHVM image scaler) for debugging. [production]
17:05 <krenair> Synchronized php-1.26wmf12/extensions/Collection/RenderingAPI.php: https://gerrit.wikimedia.org/r/#/c/222616/ - hoping this fixes T104708 (duration: 00m 44s) [production]
15:35 <YuviPanda> mount /dev/mapper/backup-others--20150703 /srv/backup-others-20150703/ on labstore2001 [production]
15:34 <YuviPanda> mkdir /srv/backup-others-20150703 on labstore2001 [production]
15:34 <YuviPanda> mkfs -t ext4 /dev/mapper/backup-others--20150703 on labstore2001 completed [production]
15:33 <YuviPanda> run mount -o ro /dev/mapper/labstore-others--20150703 /mnt/backup/others-20150703/ on labstore1002 [production]
15:32 <YuviPanda> run mkdir /mnt/backup/others-20150703 on labstore1002 [production]
15:31 <YuviPanda> run lvcreate -L 640G -s -n others-20150703 labstore/others on labstore1002 [production]
15:29 <YuviPanda> running mkfs -t ext4 /dev/mapper/backup-others--20150703 on labstore2001 [production]
15:28 <YuviPanda> run lvcreate -L 3.5T -n others-20150703 backup on labstore2001 [production]
15:25 <YuviPanda> begin process of backing up others (all labs projects except tools) on to labstore2001 from labstore1002 [production]
14:06 <jynus> Synchronized wmf-config/db-eqiad.php: repool db1022 (low traffic) (duration: 00m 54s) [production]
13:27 <jynus> Synchronized wmf-config/db-codfw.php: repool db2047 after maintenance (duration: 00m 22s) [production]
13:27 <YuviPanda> interrupting tar |ssh | tar script and cleaning out destination again [production]
13:17 <YuviPanda> clean out tar | ssh | tar target on labstore2001 [production]
13:15 <YuviPanda> /dev/null filled up on labstore1002, aborting pipe of valuable user data into it. [production]
13:13 <YuviPanda> run cd /mnt/backup/tools-20150703/ ; tar --acls --xattrs -cpf - . | pv -L 80M -C -p -r -e -b -t -B 32M -T > /dev/null on labstore1002 [production]
13:02 <YuviPanda> interrupt tar | ssh | tar on labstore1002 and killed dest on labstore2001 [production]
12:43 <mobrovac> restbase deploying restbase/deploy @ 1a826a5 [production]
12:42 <YuviPanda> interrupt tar | ssh | tar on labstore1002, clean out destination on labstore2001 [production]
12:37 <YuviPanda> interrupted tar | ssh | tar on labstore1002 and cleaned out dest on labstore2001 [production]
12:33 <YuviPanda> rm -rf /srv/backup-tools-20150703/* on labstore2001 [production]
12:31 <mark> labstore2001: mount /srv/backup -o remount,ro [production]
12:31 <YuviPanda> interrupt tar | ssh | tar on labstore1002 [production]
12:29 <YuviPanda> cd /mnt/backup/tools-20150703/ ; tar --acls --xattrs -cpf - . | ssh -i ~/.ssh/id_labstore root@labstore2001.codfw.wmnet "pv -L 80M -p -r -e -b -t -B 16M | tar --acls --xattrs -xpf - -C /srv/backup-tools-20150703" on labstore1002 [production]
12:28 <YuviPanda> cd /mnt/backup/tools-20150703/ ; tar --acls --xattrs cpf - . | ssh -i ~/.ssh/id_labstore root@labstore2001.codfw.wmnet "pv -L 80M -p -r -e -b -t -B 16M | tar --acls --xattrs xpf - -C /srv/backup-tools-20150703" on labstore1002 [production]
12:09 <YuviPanda> running mount -o ro /dev/mapper/labstore-tools--20150703 /mnt/backup/tools-20150703/ now [production]
11:57 <YuviPanda> run lvcreate -L 640G -s -n tools-20150703 labstore/tools on labstore1002 [production]
11:50 <YuviPanda> running lvcreate -L 640G -s tools -n tools-20150703 labstore on labstore1002 [production]
11:26 <YuviPanda> umount /mnt/backup/project/tools/ on labstore1002 [production]
11:24 <YuviPanda> ran mount /dev/mapper/backup-tools--20150703 /srv/backup-tools-20150703/ on labstore2001 [production]
11:22 <YuviPanda> mkdir /srv/backup-tools-20150703 on labstore2001 [production]
11:13 <YuviPanda> run mkfs -t ext4 /dev/mapper/backup-tools--20150703 on labstore2001 [production]
11:09 <YuviPanda> lvcreate -L 6TB -n tools-20150703 backup on labstore2001 [production]
11:09 <jynus> reimports finished on dbstore2* hosts and puppet reenabled after T104471 was fixed [production]
10:56 <mobrovac> restbase disabling puppet on restbase1005 to tweak JVM params for cassandra [production]
10:50 <YuviPanda> started du of maps project on labstore2001 [production]
09:36 <mobrovac> restbase restarting cassandra on rb1002 [production]