2301-2350 of 10000 results (59ms)
2019-02-11 §
07:00 <marostegui> Restart icinga on icinga1001 - checks went awol [production]
06:51 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1079 (duration: 00m 48s) [production]
06:14 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1079 (duration: 00m 48s) [production]
06:14 <marostegui@deploy1001> sync-file aborted: Depool db0179 (duration: 00m 01s) [production]
04:23 <TimStarling> on mwmaint1002: running normalizeThrottleParameters.php --dry-run on all wikis (T209565) [production]
04:19 <tstarling@deploy1001> Synchronized php-1.33.0-wmf.16/extensions/AbuseFilter/maintenance/normalizeThrottleParameters.php: maintenance script update for new dry run (duration: 00m 47s) [production]
04:19 <tstarling@deploy1001> Synchronized php-1.33.0-wmf.16/extensions/WikimediaEvents/tests/phpunit/PageViewsTest.php: test-only undeployed change (duration: 00m 46s) [production]
04:18 <tstarling@deploy1001> Synchronized php-1.33.0-wmf.16/extensions/NavigationTiming/tests/ext.navigationTiming.test.js: test-only undeployed change (duration: 00m 51s) [production]
04:10 <tstarling@deploy1001> sync-file aborted: test-only undeployed change (duration: 00m 12s) [production]
03:05 <kartik@deploy1001> Finished deploy [cxserver/deploy@ee4a15a]: Update cxserver to 8928852 (T213256) (duration: 04m 08s) [production]
03:00 <kartik@deploy1001> Started deploy [cxserver/deploy@ee4a15a]: Update cxserver to 8928852 (T213256) [production]
2019-02-10 §
19:59 <volans|off> force rebooting mw1299, stuck again - T215569 [production]
19:16 <volans|off> forcing reboot of icinga1001 because it's stuck again (no ping, no ssh, CPU stuck messages on console) - T214760 [production]
09:25 <marostegui> Disable notifications for lag checks on dbstore1002 - T210478 [production]
2019-02-09 §
21:42 <Reedy> running `foreachwiki refreshImageMetadata.php --mediatype BITMAP --mime image/vnd.djvu --force` on mwmaint1002 T215635 [production]
21:41 <Reedy> refreshImageMetadata.php for commonswiki done T215635 [production]
16:51 <Jeff_Green> restarted icinga process on icinga1001 because of passive check alert-storm [production]
2019-02-08 §
23:23 <Reedy> running `refreshImageMetadata.php --mediatype BITMAP --mime image/vnd.djvu --force` against commonswiki on mwmaint1002 T215635 (this time we mean it) [production]
22:56 <Reedy> running `refreshImageMetadata.php --mediatype BITMAP --mime image/vnd.djvu` against commonswiki on mwmaint1002 T215635 [production]
21:25 <reedy@deploy1001> Synchronized multiversion/MWMultiVersion.php: Move variable (duration: 00m 49s) [production]
19:50 <krinkle@deploy1001> Synchronized w/touch.php: Ia1e610a5f (duration: 00m 46s) [production]
19:49 <krinkle@deploy1001> Synchronized w/robots.php: Ia1e610a5f (duration: 00m 46s) [production]
19:48 <krinkle@deploy1001> Synchronized w/favicon.php: Ia1e610a5f (duration: 00m 46s) [production]
19:47 <krinkle@deploy1001> Synchronized w/extract2.php: Ia1e610a5f (duration: 00m 48s) [production]
18:14 <gtirloni> T213527 graphite2002 disabled puppet and commented prometheus_puppet_agent_stats cronjob due to cronspam [production]
18:08 <jynus@deploy1001> Synchronized wmf-config/db-eqiad.php: Increase weight for s1 rc slaves (duration: 00m 49s) [production]
17:54 <mutante> phab1001 - restart aphlict service [production]
17:52 <mutante> phab1001 - restarting phd service [production]
17:49 <arturo> T215605 add prometheus-openstack-exporter 0.0.8-4 to stretch-wikimedia [production]
17:47 <mutante> phab1001 - restarting apache2 service for library upgrade [production]
17:42 <mutante> graceful reload of apache on phabricator prod server (phab1001) [production]
17:27 <XioNoX> merge Icinga: add ping check for ulsfo PDUs [production]
16:50 <ejegg> updated payments-wiki-staging from 52a271e681 to 31647bc97e [production]
16:09 <jynus> stopping s1 replication on dbstore1001 to speed up cloning T214720 [production]
16:08 <moritzm> imported git-fat 0.1.3-2+deb10u1 to buster-wikimedia (T213527) [production]
15:46 <marostegui> Repool labsdb1009 - T212308 [production]
15:33 <_joe_> apt-get upgrade on mwmaint2001 to fix the php installation T215376 [production]
15:31 <moritzm> imported debmonitor 0.1.5-1+deb10u1 to buster-wikimedia (T213527) [production]
15:31 <_joe_> upgraded all php extensions to php 7.2 compatible versions on mwmaint1002 [production]
15:10 <jijiki> Upgrading php-redis 4.1.1 to mwmaint1002 - T215376 [production]
14:51 <marostegui> Reload haproxy on dbproxy1011 to depool labsdb1009 - https://phabricator.wikimedia.org/T212308 [production]
13:56 <moritzm> updated firmware-enriched buster netboot image to 20190208 daily build, the alpha5 image no longer works as Linux 4.19.16-1 bumped the ABI and migrated to testing yesterday [production]
13:45 <jynus> racadm serveraction powercycle db1114 [production]
13:39 <onimisionipe> starting osm-initial-import for maps2004 which is the newly migrated to stretch master - T198622 [production]
13:37 <elukey> roll restart of aqs on aqs1* to pick up new druid backend changes [production]
13:05 <arturo> T209029 reimaging cloudelastic1004 [production]
12:54 <ejegg> updated fundraising CiviCRM from 3a1bb82373 to a541a83cb2 [production]
12:51 <jynus> disabling notifications on db1114 [production]
12:44 <elukey@deploy1001> Synchronized wmf-config/db-eqiad.php: depooling db1114, host down (duration: 00m 47s) [production]
11:36 <moritzm> reimage graphite2002 to buster [production]