1301-1350 of 10000 results (66ms)
2018-06-30 §
18:15 <chicocvenancio> pushed new config to PAWS to fix dumps nfs mountpoint [tools]
16:40 <zhuyifei1999_> because tools-paws-master-01 was having ~1000 loadavg due to NFS having issues and processes stuck in D state [tools]
16:39 <zhuyifei1999_> reboot tools-paws-master-01 [tools]
16:35 <zhuyifei1999_> `root@tools-paws-master-01:~# sed -i 's/^labstore1006.wikimedia.org/#labstore1006.wikimedia.org/' /etc/fstab` [tools]
16:34 <andrewbogott> "sed -i '/labstore1006/d' /etc/fstab" everywhere [tools]
16:28 <chasemp> labstore1006:~# service nfs-kernel-server restart [production]
2018-06-29 §
23:49 <bd808> Stopped webservice due to CrashLoopBackOff failures. Caused by "ImportError: No module named 'yamlconf'" [tools.ores-support-checklist]
23:44 <bd808> test [tools.stashbot]
21:37 <chasemp> (late log) of VLAN creations for T184209 cloud-instances2-b-eqiad and cloud-instance-transport1-b-eqiad [production]
19:11 <herron> stat1005:~# killall -u ironholds T197895 [production]
17:41 <bd808> Rescheduling continuous jobs away from tools-exec-1408 where load is high [tools]
17:28 <XioNoX> Re-activating v6 BGP session to Telia on cr2-eqiad - T198502 [production]
17:21 <XioNoX> deactivating v6 BGP session to Telia on cr2-eqiad - T198502 [production]
17:11 <bd808> Rescheduled jobs away from toole-exec-1404 where linkwatcher is currently stealing most of the CPU (T123121) [tools]
16:46 <bd808> Killed orphan tool owned processes running on the job grid. Mostly jembot and wsexport php-cgi processes stuck in deadlock following an OOM. T182070 [tools]
14:43 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1092 after alter table (duration: 00m 50s) [production]
12:30 <moritzm> uploaded ffmpeg 3.2.10-1~deb9u1+wmf1 to apt.wikimedia.org/stretch-wikimedia (component/vp9) (linked against libvpx 1.7 and with backported row-mt support) (T190333) [production]
11:11 <moritzm> installing zsh updates from jessie 8.11 point release [production]
11:01 <moritzm> installing lame security updates [production]
10:34 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2058 after reimage (duration: 00m 51s) [production]
10:21 <mobrovac@deploy1001> Finished deploy [citoid/deploy@40cdff7]: Update citoid to fd77117 - T165105 T197853 (duration: 03m 26s) [production]
10:20 <marostegui> Deploy schema change on db1092 T191316 T192926 T89737 T195193 [production]
10:19 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1092 for alter table (duration: 00m 50s) [production]
10:18 <mobrovac@deploy1001> Started deploy [citoid/deploy@40cdff7]: Update citoid to fd77117 - T165105 T197853 [production]
10:14 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1104 after alter table (duration: 00m 50s) [production]
09:23 <marostegui> Stop MySQL on db2058 to reimage it [production]
09:23 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2058 for reimage (duration: 00m 50s) [production]
09:18 <moritzm> uploaded libvpx 1.7.0-3+wmf1 to apt.wikimedia.org/stretch-wikimedia (component/vp9) (T190333) [production]
09:08 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2054 after reimage (duration: 00m 51s) [production]
08:39 <vgutierrez> Apply new TLS varnishkafka settings in cache::text nodes - T182993 [production]
08:27 <akosiaris> upgrade php on phab1001 to 5.6.33+dfsg-0+deb8u1 [production]
08:03 <jynus@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool es1016 (duration: 00m 50s) [production]
07:53 <marostegui> Stop MySQL on db2054 for reimage [production]
07:53 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2054 for reimage (duration: 00m 50s) [production]
06:21 <jynus> stop and reimage es1016 [production]
05:40 <elukey> force umount of dumps labstore nfs mountpoints on stat100[56]/notebook100[34] to reduce load (also too many open files) [production]
05:02 <marostegui> Deploy schema change on db1104 T191316 T192926 T89737 T195193 [production]
05:01 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1104 for alter table (duration: 00m 50s) [production]
05:00 <marostegui> /script unload irssinotifier [production]
04:55 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1101:3318 after alter table (duration: 00m 50s) [production]
04:52 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool es1015 (duration: 00m 52s) [production]
01:53 <catrope@deploy1001> Finished scap: Full scap to fix i18n issues on testwiki (duration: 32m 42s) [production]
01:20 <catrope@deploy1001> Started scap: Full scap to fix i18n issues on testwiki [production]
01:19 <krinkle@deploy1001> Synchronized vendor/: I39e592d837 - remove redundant composer packages for xhgui profiling (duration: 00m 52s) [production]
2018-06-28 §
23:54 <catrope@deploy1001> Synchronized php-1.32.0-wmf.10/resources/src/mediawiki.rcfilters/: Watchlist perf fixes (T198359, T198399) (duration: 00m 52s) [production]
23:18 <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Increase Schema:CitationUsage sampling rate to 100% (T191086) (duration: 00m 51s) [production]
21:55 <niharika29@deploy1001> Synchronized php-1.32.0-wmf.10/extensions/PageTriage/: Update extension directory (duration: 00m 51s) [production]
21:52 <niharika29@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable Draft namespace and AfC mode for PageTriage on testwiki T198143 (duration: 00m 53s) [production]
21:17 <hasharAway> castor02: nuking cache of npm/node jobs via rm -fR /srv/jenkins-workspace/caches/*/*/*node* (note: other jobs might still have a npm cache) | T198348 [releng]
21:09 <hasharAway> castor: nuking caches castor-mw-ext-and-skins/master/wmf-quibble-vendor-mysql-hhvm-docker/npm and castor-mw-ext-and-skins/master/wmf-quibble-vendor-mysql-php70-docker/npm | T198348 [releng]