1451-1500 of 10000 results (37ms)
2016-03-28 §
20:30 <chasemp> change perm grant files from create-dbusers for chmod 400 chat chattr +i [tools]
18:30 <andrewbogott> labs public DNS will remain in a transitional state until MarkMonitor updates their records and the TTL expires. DNS resolution should proceed as usual in the meantime. [production]
18:03 <mutante> mw1140 restart hhvm [production]
17:52 <cmjohnson1> shutting down radon to re-apply thermal paste: [production]
17:45 <mutante> mw1152,mw2090,terbium apt-get remove python-mysqldb (T84075) [production]
17:44 <legoktm> manually fixed T129881 rename [production]
17:20 <paravoid> cr1/2-eqiad: deactivating static routes for ns0/ns1 (ipv4/ipv6) pointing to radon [production]
17:18 <bblack> rebooting radon again [production]
15:31 <thcipriani@tin> Synchronized php-1.27.0-wmf.18/extensions/OpenStackManager/OpenStackManager.php: SWAT: Wikitech: Remove address, domain, proxy special pages [[gerrit:279569]] (duration: 00m 31s) [production]
13:32 <bblack> rebooting radon (unresponsive console) [production]
08:44 <paravoid> force-rebooting ms-be2008 to fix hundreds of unkillable mkfs.xfs stuck due to a hosed disk [production]
08:21 <paravoid> powercycling ms-be2016, down for ~1d, unresponsive on console [production]
02:35 <l10nupdate@tin> ResourceLoader cache refresh completed at Mon Mar 28 02:35:00 UTC 2016 (duration 8m 26s) [production]
02:26 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.18) (duration: 11m 32s) [production]
2016-03-27 §
22:21 <ori@tin> Synchronized wmf-config/StartProfiler.php: I1b5c620b85: Better request profiling via XWD header (duration: 00m 33s) [production]
20:11 <elukey> restarted hhvm on mw1123 [production]
20:04 <elukey> Increased nf_conntrack_max to ~528k for the kafka brokers (https://gerrit.wikimedia.org/r/279776) [production]
02:57 <jzerebecki> disabling 2fa for Dereckson T130892 [labswiki]> delete from oathauth_users where id=402; [production]
02:31 <l10nupdate@tin> ResourceLoader cache refresh completed at Sun Mar 27 02:31:03 UTC 2016 (duration 8m 35s) [production]
02:22 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.18) (duration: 10m 06s) [production]
01:51 <jzerebecki> disabling 2fa for Hym411 T130994 [labswiki]> delete from oathauth_users where id=1363; [production]
2016-03-26 §
22:38 <jzerebecki> reloading zuul for 2d7e050..46923c8 [releng]
21:11 <csteipp_ooo> removed 2FA from wikitech accounts that looked to be affected by T130892 [production]
16:44 <Luke081515> updating all git repos [rcm.cac]
16:21 <krenair@tin> Synchronized php-1.27.0-wmf.18/extensions/SemanticForms: https://gerrit.wikimedia.org/r/#/c/279701/ (duration: 00m 32s) [production]
13:46 <urandom> Cassandra on restbase2004.codfw.wmnet shut down, hardware failure; Down for the weekend : T130990 [production]
13:26 <urandom> Stopping Cassandra on restbase2004.codfw.wmnet [production]
02:31 <l10nupdate@tin> ResourceLoader cache refresh completed at Sat Mar 26 02:31:13 UTC 2016 (duration 8m 34s) [production]
02:22 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.18) (duration: 10m 06s) [production]
00:11 <mutante|away> scb[12]00[12] - delete changeprop main.log per request [production]
2016-03-25 §
23:57 <bd808> Removed self (BryanDavis) from project [xtools]
23:56 <bd808> Deleted stale puppet lock file on xtools-lb [xtools]
23:55 <marxarelli> deleting instances integration-slave-trusty-1002 and integration-slave-trusty-1005 [releng]
23:54 <bd808> Added self (BryanDavis) as admin to debug puppet failure [xtools]
23:54 <marxarelli> deleting jenkins nodes integration-slave-trusty-1002 and integration-slave-trusty-1005 [releng]
23:41 <marxarelli> completed rolling manual deploy of https://gerrit.wikimedia.org/r/#/c/279640/ to trusty slaves [releng]
23:27 <marxarelli> starting rolling offline/remount/online of trusty slaves to increase tmpfs size [releng]
23:22 <marxarelli> pooled new trusty slaves integration-slave-trusty-1024 and integration-slave-trusty-1025 [releng]
23:13 <jzerebecki> reloading zuul fro 0aec21d..2d7e050 [releng]
22:14 <marxarelli> creating new jenkins node for integration-slave-trusty-1024 [releng]
22:11 <marxarelli> rebooting integration-slave-trusty-{1024,1025} before pooling as replacements for trusty-1002 and trusty-1005 [releng]
21:52 <ori> Updated Grafana to latest nightly [production]
21:19 <ori@tin> Synchronized wmf-config/logging.php: Ibc96f9d3bd: Integrate X-Wikimedia-Debug with Logstash; I28b137f5e035: Logging: convert to short array syntax (duration: 00m 36s) [production]
21:06 <marxarelli> repooling integration-slave-trusty-{1005,1002} to help with load while replacement instances are provisioning [releng]
20:46 <ori@tin> Synchronized php-1.27.0-wmf.18/includes/filerepo/LocalRepo.php: Idaa1237638: Request-local caching of image_redirect and I545ce6b160b: Lower pcTTL in checkRedirect() to 30 (duration: 00m 37s) [production]
20:23 <Krenair> started redis-server on deployment-redis01 [deployment-prep]
20:23 <Krenair> repaired centralauth.spoofuser table on deployment-db1 [deployment-prep]
20:23 <Krenair> fiddled around with puppet on deployment-cache-text04 earlier to fix certs etc. [deployment-prep]
19:31 <tgr> running extensions/CentralAuth/maintenance/checkLocal{Names,User}.php on jawiktionary and dewikiquote [production]
19:08 <mutante> scb1001 - Unit changeprop.service entered failed state - it's a new deployment though, acked earlier by mobrovac already [production]