1201-1250 of 10000 results (48ms)
2017-04-03 §
02:24 <l10nupdate@tin> scap sync-l10n completed (1.29.0-wmf.18) (duration: 09m 39s) [production]
2017-04-02 §
08:18 <elukey> powercycle ms-be1016 (stuck in console, answers pings but not ssh) [production]
07:25 <ariel@tin> Finished deploy [dumps/dumps@1ac3fb3]: var/method name cleanups, refactor, pregenerate page ranges for page content jobs, auto retry of failed page ranges (duration: 00m 03s) [production]
07:25 <ariel@tin> Started deploy [dumps/dumps@1ac3fb3]: var/method name cleanups, refactor, pregenerate page ranges for page content jobs, auto retry of failed page ranges [production]
02:24 <l10nupdate@tin> scap sync-l10n completed (1.29.0-wmf.18) (duration: 09m 44s) [production]
2017-04-01 §
19:01 <elukey> restart hhvm on mw1191 (dump debug in /tmp/hhvm.16619.bt.) - threads stuck in HPHP::Treadmill::getAgeOldestRequest [production]
15:28 <andrewbogott> added five new exec nodes, tools-exec-1425 through 1429 [tools]
14:26 <chasemp> up nfs thresholds https://gerrit.wikimedia.org/r/#/c/345975/ [tools]
14:00 <chasemp> disable puppet on tools-grid-msater [tools]
13:52 <chasemp> tools-grid-master tc-setup clean [tools]
13:40 <chasemp> restart nscd and nscld on tools-grid-master [tools]
13:31 <chasemp> reboot tools-exec-1420 [tools]
09:48 <Sagan> puppet on deployment-tin looks like it is not running properly [releng]
07:26 <joal> Kill old cassandra bundles and restart new one for project_v2 production codeb [analytics]
02:37 <l10nupdate@tin> ResourceLoader cache refresh completed at Sat Apr 1 02:37:30 UTC 2017 (duration 5m 20s) [production]
02:32 <l10nupdate@tin> scap sync-l10n completed (1.29.0-wmf.18) (duration: 13m 27s) [production]
2017-03-31 §
23:11 <mutante> ruthenium: logrotate --force /etc/logrotate.d/parsoid (note this is existing file "parsoid" not new file "parsoid_testing") (T161920) [production]
22:25 <yuvipanda> apt-get update && apt-get install kubernetes-node on tools-proxy-01 to upgrade kube-proxy systemd service unit [tools]
20:18 <elukey> stopping jobrunners on mw116[89] and restarting hhvm after https://gerrit.wikimedia.org/r/345881 [production]
19:44 <Reedy> Stop badge hacks from messing up the entire page on IE 11 on MonoBook T161869 [production]
19:42 <reedy@tin> Synchronized php-1.29.0-wmf.18/extensions/Echo: Stop badge hacks from messing up the entire page on IE 11 on MonoBook T161689 (duration: 00m 50s) [production]
19:16 <mutante> ruthenium also deleting ancient "htmldumper" data, gwicke confirmed it's not needed anymore [production]
18:27 <mutante> ruthenium mounting /dev/mapper/ruthenium--vg-tank into /srv/visualdiff/pngs | deleted "mysql" and "dumps" data that was on previously unmounted partition , subbu checked that wasn't needed anymore, we still need logrotate (T161920) [production]
18:14 <mutante> ruthenium mounting /dev/mapper/ruthenium--vg-tank which wasnt used at all.. bam.. over 477GB of free space [production]
16:02 <volans@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=codfw,name=ms-fe2008.codfw.wmnet [production]
16:00 <volans@puppetmaster1001> conftool action : set/pooled=no; selector: dc=codfw,name=ms-fe2008.codfw.wmnet [production]
15:59 <volans@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=codfw,name=ms-fe2007.codfw.wmnet [production]
15:57 <volans@puppetmaster1001> conftool action : set/pooled=no; selector: dc=codfw,name=ms-fe2007.codfw.wmnet [production]
15:55 <mobrovac@tin> Finished deploy [trending-edits/deploy@26b5eb4]: Config change: lower min_edits to 15 T160127 (duration: 06m 37s) [production]
15:55 <volans@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=codfw,name=ms-fe2006.codfw.wmnet [production]
15:52 <volans@puppetmaster1001> conftool action : set/pooled=no; selector: dc=codfw,name=ms-fe2006.codfw.wmnet [production]
15:49 <mobrovac@tin> Started deploy [trending-edits/deploy@26b5eb4]: Config change: lower min_edits to 15 T160127 [production]
15:44 <volans@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=codfw,name=ms-fe2005.codfw.wmnet [production]
15:22 <volans@puppetmaster1001> conftool action : set/pooled=true; selector: dnsdisc=imagescaler-rw,name=eqiad [production]
15:17 <volans@puppetmaster1001> conftool action : set/pooled=false; selector: dnsdisc=imagescaler-rw,name=eqiad [production]
15:02 <volans@puppetmaster1001> conftool action : set/pooled=no; selector: dc=codfw,name=ms-fe2005.codfw.wmnet [production]
15:01 <oblivian@puppetmaster1001> conftool action : set/ttl=300; selector: dnsdisc=restbase-async [production]
15:01 <oblivian@puppetmaster1001> conftool action : set/pooled=false; selector: dnsdisc=restbase-async,name=eqiad [production]
14:58 <oblivian@puppetmaster1001> conftool action : set/pooled=true; selector: dnsdisc=restbase-async,name=eqiad [production]
14:56 <oblivian@puppetmaster1001> conftool action : set/ttl=10; selector: dnsdisc=restbase-async [production]
14:55 <_joe_> reducing ttl on the restbase-async discovery record, then flipping eqiad to active [production]
14:55 <volans> deploying the use of discovery URL to swift-proxy hosts in codfw T160178#3136906 [production]
14:09 <_joe_> performing a rolling restart of changeprop after puppet runs on scb* [production]
14:00 <oblivian@puppetmaster1001> conftool action : set/pooled=true; selector: dnsdisc=restbase-async,name=codfw [production]
13:42 <andrewbogott> deleting shutoff precise instances 'wlm-mysql-master' and 'wlm-apache1' [wlmjudging]
13:41 <andrewbogott> deleting shutoff precise instance 'dannyb' [wildcat]
13:41 <andrewbogott> deleting shutoff precise instance 'wikistream-web' [wikistream]
13:40 <andrewbogott> deleting shutoff precise instance 'wsexport' [wikisource-tools]
13:39 <andrewbogott> deleting shutoff precise instance 'wikisource-dev' [wikisource-dev]
13:38 <andrewbogott> deleting shutoff precise instance wikidata-wdq-mm [wikidata-dev]