1051-1100 of 10000 results (39ms)
2017-07-19 §
10:06 <marostegui> Deploy alter table on s1 - db1051 - T166204 [production]
10:04 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1051 - T166204 (duration: 00m 47s) [production]
10:01 <filippo@tin> Finished deploy [statsv/statsv@0a86be8]: (no justification provided) (duration: 00m 03s) [production]
10:01 <filippo@tin> Started deploy [statsv/statsv@0a86be8]: (no justification provided) [production]
09:58 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1065 - T166204 (duration: 00m 47s) [production]
09:16 <ema> finish up codfw cache_text/upload varnish/kernel upgrades [production]
09:05 <oblivian@neodymium> conftool action : set/pooled=false; selector: name=eqiad,dnsdisc=(restbase-async|citoid) [production]
09:03 <XioNoX> codfw repooled in dns - T170380 [production]
09:01 <hashar> restarting nodepool for upgrade 0.1.1-wmf7 -> 0.1.1-wmf8 [production]
08:48 <moritzm> uploaded nodepool 0.1.1+wmf8 to apt.wikimedia.org [production]
08:29 <XioNoX> asw-c-codfw back online - T170380 [production]
08:28 <XioNoX> asw-c-codfw restarted 8min ago for switch upgrade - T170380 [production]
08:09 <akosiaris> disable librenms crons on netmon1002 for a while [production]
07:57 <marostegui> Drop migrateuser_medium from s7 - T170310 [production]
07:49 <oblivian@neodymium> conftool action : set/pooled=true; selector: name=eqiad,dnsdisc=(restbase-async|citoid) [production]
05:26 <_joe_> ran systemctl reset-failed on codfw jobrunners after the jobrunner process was activated by mistake running scap at 21.20 UTC yesterday [production]
03:03 <l10nupdate@tin> LocalisationUpdate failed: git pull of extensions failed [production]
01:27 <mutante> netmon1001 - stopping all the services, killing snmpwalk, disarming keyholder [production]
00:35 <reedy@tin> Synchronized wmf-config/CommonSettings.php: Remove rcs1001 and rcs1002 from CommonSettings wgRCFeeds. Stops a load of logspam T170157 (duration: 00m 48s) [production]
2017-07-18 §
23:53 <mutante> netmon1002 - copied Letsencrypt cert/key for librenms from netmon1001 for migration after netmon1002 has been reinstalled and now has RAID. (T159756) [production]
23:40 <thcipriani@tin> Synchronized wmf-config/InterwikiSortOrders.php: SWAT: [[gerrit:365451|Add din to InterwikiSortOrders]] T168518 (duration: 00m 46s) [production]
23:35 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:365942|Add Welsh mobile logo (just changes 'k' to 'c']] PART II (duration: 00m 46s) [production]
23:34 <thcipriani@tin> Synchronized static/images/mobile/copyright/wikipedia-wordmark-cy.svg: SWAT: [[gerrit:365942|Add Welsh mobile logo (just changes 'k' to 'c']] PART I (duration: 00m 47s) [production]
23:27 <thcipriani@tin> Synchronized php-1.30.0-wmf.9/extensions/Thanks/extension.json: SWAT: [[gerrit:366168|Add missing jQueryMsg dependency for mobile diff view]] T170917 (duration: 00m 47s) [production]
23:22 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:360371|Enable OOjs UI EditPage buttons on all Wikipedias]] T162849 (duration: 00m 47s) [production]
23:13 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:365884|Enable CodeMirror on simplewiki for better testing and more exposure]] (duration: 00m 48s) [production]
22:58 <thcipriani> restared jobrunner on mw1299.eqiad.wmnet mw1168.eqiad.wmnet mw1164.eqiad.wmnet mw1305.eqiad.wmnet mw1304.eqiad.wmnet mw1301.eqiad.wmnet mw1259.eqiad.wmnet mw1166.eqiad.wmnet mw1300.eqiad.wmnet [production]
22:42 <krinkle@tin> Finished deploy [jobrunner/jobrunner@5f6099f]: (no justification provided) (duration: 08m 18s) [production]
22:34 <krinkle@tin> Started deploy [jobrunner/jobrunner@5f6099f]: (no justification provided) [production]
22:02 <krinkle@tin> Finished deploy [jobrunner/jobrunner@5f6099f]: (no justification provided) (duration: 07m 58s) [production]
21:54 <krinkle@tin> Started deploy [jobrunner/jobrunner@5f6099f]: (no justification provided) [production]
21:43 <Krinkle> Attempt to deploy mediawiki/services/jobrunner – https://gerrit.wikimedia.org/r/#/c/349364/ - failed. [production]
19:56 <dzahn@neodymium> conftool action : set/pooled=yes; selector: name=mw2202.codfw.wmnet [production]
19:48 <robh> starting wipe on cp400[1-4] per T169020 [production]
19:15 <demon@tin> rebuilt wikiversions.php and synchronized wikiversions files: group0 to wmf.10 [production]
18:59 <demon@tin> Synchronized php-1.30.0-wmf.9/extensions/MobileFrontend/extension.json: One (more) last thing (duration: 02m 49s) [production]
18:51 <demon@tin> Synchronized php-1.30.0-wmf.9/extensions/MobileFrontend/extension.json: One last thing (duration: 02m 55s) [production]
18:42 <mutante> netmon1002 - reinstall OS - didn't use the right partman recipe - didn't have md0 - revoke old puppet cert , salt-key, scheduled downtime, services over at netmon2001 [production]
18:36 <mutante> mw2202 - scheduled downtime - mainboard replacement [production]
18:36 <ejegg> updated payments-wiki from bdc52265d78c55cfc6a732f14519f5f79c9d2d94 to c3be2bfd8f2b9f9eac4c80b45096713c7fdcceff [production]
18:29 <demon@tin> Finished scap: mobilefrontend wmf.9 + forced l10n rebuild (duration: 20m 53s) [production]
18:26 <mutante> mw2202 - remove /etc/udev/rules.d/70-persistent-net.rules for mainboard replacement - to detect new NICs with new MACs (T170307) [production]
18:24 <dzahn@neodymium> conftool action : set/pooled=no; selector: name=mw2202.codfw.wmnet [production]
18:08 <demon@tin> Started scap: mobilefrontend wmf.9 + forced l10n rebuild [production]
18:02 <ottomata> stopping kafka on kafka1012 again, i think we swapped the wrong disk T168927 [production]
17:55 <awight@tin> Finished deploy [ores/deploy@1d35aa5]: T170485 (duration: 35m 06s) [production]
17:47 <mutante> smokeping - switched to netmon2001 - ping times to codfw hosts went down - ping times to eqiad hosts went up - since service is on both but data has been synced over [production]
17:41 <demon@tin> Synchronized wmf-config/InitialiseSettings.php: labtest typofix for tgr (duration: 00m 46s) [production]
17:21 <mobrovac@tin> Finished deploy [parsoid/deploy@1eaa07e]: Bring wtp2019 up to date and repool it - T146113 (duration: 01m 02s) [production]
17:20 <mobrovac@tin> Started deploy [parsoid/deploy@1eaa07e]: Bring wtp2019 up to date and repool it - T146113 [production]