1451-1500 of 10000 results (37ms)
2017-04-18 §
10:26 <switchdc> (volans@sarin) START TASK - switchdc.stages.t00_disable_puppet(codfw, eqiad) Disabling puppet on selected hosts [production]
10:25 <volans> Final test of switchdc steps in the codfw->eqiad configuration, only idempotent changes, T160178 [production]
10:25 <moritzm> installing wireshark security updates [production]
10:20 <moritzm> uploaded HHVM 3.18.2+wmf2 for jessie-wikimedia/experimental (includes fix for T162354) [production]
09:52 <oblivian:> Setting zotero in codfw UP [production]
09:50 <_joe_> testing switchover script for services, will act on zotero in codfw [production]
09:45 <_joe_> adding 60G to the ocg output partition on ocg1003 [production]
09:17 <oblivian@neodymium> conftool action : set/pooled=true; selector: dnsdisc=zotero,name=codfw [production]
09:03 <volans> upgrading conftool to v0.4.1 on neodymium/sarin [production]
07:48 <_joe_> uploaded python-conftool 0.4.1 to jessie-wikimedia [production]
07:42 <_joe_> cleaning up orphaned COW images in /var/cache/pbuilder/build/ on copper [production]
06:16 <marostegui> For the record: restarted s7 instance on db1069 - T163183 [production]
00:36 <catrope@tin> Synchronized php-1.29.0-wmf.20/extensions/MobileFrontend/resources/mobile.mainMenu/mainmenu.less: T163059 (duration: 03m 07s) [production]
2017-04-17 §
23:37 <mutante> runnin rmmod acpi_pad on the 16 R320 via cumin, since blacklisting in puppet does not actively remove, confirmed unloaded. (16/16) success ratio (>= 100.0% threshold) for command: 'lsmod|grep -c acpi_pad ||:' (T162850) [production]
23:33 <mutante> running puppet via cumin on all 16 Dell PowerEdge R320, adding blacklist file for acpi_pad kernel module. 15/16 success, all but tin (T162850) [production]
22:46 <catrope@tin> Synchronized php-1.29.0-wmf.20/extensions/WikimediaEvents/modules/ext.wikimediaEvents.recentChangesClicks.js: T158458 T163152 (duration: 03m 01s) [production]
22:42 <mutante> tin - load average going down, acpi_pad processes gone, cpu usage low again (T163158) [production]
22:40 <mutante> tin - rmmod acpi_pad (T163158) [production]
22:08 <catrope@tin> Synchronized php-1.29.0-wmf.20/extensions/WikimediaEvents/modules/ext.wikimediaEvents.recentChangesClicks.js: T158458 T163152 (duration: 16m 23s) [production]
19:16 <mutante> tegmen test ircecho stop/start service to confirm it's fine on jessie/prod icinga role (that's the passive server) [production]
19:02 <demon@tin> Synchronized wmf-config/: Pruning some old extension message files, co-master sync (duration: 01m 52s) [production]
18:58 <demon@tin> Pruned MediaWiki: 1.29.0-wmf.15 (duration: 00m 14s) [production]
18:46 <maxsem@tin> Finished deploy [tilerator/deploy@001811e]: https://gerrit.wikimedia.org/r/#/c/348224/ to test hosts only (duration: 00m 19s) [production]
18:46 <maxsem@tin> Started deploy [tilerator/deploy@001811e]: https://gerrit.wikimedia.org/r/#/c/348224/ to test hosts only [production]
18:45 <maxsem@tin> scap aborted: https://gerrit.wikimedia.org/r/#/c/348224/ to test hosts only (duration: 00m 19s) [production]
18:45 <maxsem@tin> Started scap: https://gerrit.wikimedia.org/r/#/c/348224/ to test hosts only [production]
15:48 <mobrovac@tin> Finished deploy [restbase/deploy@6595298]: Update client caching headers for T161284 (duration: 08m 15s) [production]
15:40 <mobrovac@tin> Started deploy [restbase/deploy@6595298]: Update client caching headers for T161284 [production]
15:34 <mobrovac@tin> Finished deploy [restbase/deploy@6595298]: (no justification provided) (duration: 01m 29s) [production]
15:33 <mobrovac@tin> Started deploy [restbase/deploy@6595298]: (no justification provided) [production]
15:32 <mobrovac@tin> Finished deploy [restbase/deploy@6595298]: (no justification provided) (duration: 01m 42s) [production]
15:31 <mobrovac@tin> Started deploy [restbase/deploy@6595298]: (no justification provided) [production]
09:33 <marostegui> Silence alerts for restbase2004 and restbase2009 T160759 [production]
2017-04-16 §
15:44 <elukey> restart ocg on ocg1003 to clean up deleted files in lsof [production]
15:35 <elukey> executing sudo find -name *.pdf -mtime +3 -exec rm {} \; on ocg1003's /srv/deployment/ocg/output to clean up some disk space - T162780 [production]
2017-04-14 §
23:14 <jynus> skipping CREATE DATABASE wbwikimedia on dbstore2001- duplicate declaration due to multi-source [production]
22:58 <jynus> skipping CREATE DATABASE pawikisource on dbstore2001- duplicate declaration due to multi-source [production]
22:49 <volans> restarting parsoid to get the disable linter change T148609 [production]
22:17 <Reedy> created linter tables on wbwikimedia T148609 [production]
22:16 <Reedy> created linter tables on pawikisource T148609 [production]
20:53 <reedy@tin> Synchronized wmf-config/InitialiseSettings.php: Disable Linter on larger wikis T148609 (duration: 00m 41s) [production]
20:26 <reedy@tin> Synchronized wmf-config/abusefilter.php: abusefilter-modify-restricted for trwiki T161960 (duration: 01m 38s) [production]
17:48 <mutante> mw1297 - restarted hhvm and apache [production]
17:07 <twentyafterfour> deployed phabricator hotfix for T162943 [production]
10:29 <elukey> rollback systctl settings on mw1306 after experiment (stop jobchron/runner, stop hhvm, restore systctl settings, restart hhvm and job* daemons) [production]
09:50 <elukey> temporarily set sysctl -w net.netfilter.nf_conntrack_max=524288 on mw1306 (jobrunner) as test - (rollback: sysctl -w net.netfilter.nf_conntrack_max=262144") [production]
09:43 <elukey> temporarily set sysctl -w net.ipv4.ip_local_port_range="15000 64000" on mw1306 (jobrunner) as test - (rollback: sysctl -w net.ipv4.ip_local_port_range="32768 60999") - T157968 [production]
08:32 <elukey> restored appendfsync to 'everysec' on Redis rdb2005:6380 (end of performance experiment) [production]
07:23 <elukey> executed CONFIG SET appendfsync no on redis2005:6780 as performance test [production]
00:39 <niharika29@tin> Synchronized wmf-config/abusefilter.php: Fix Abuse Filter configuration for tr.wikipedia (T161960) (duration: 00m 42s) [production]