| 2019-03-13
      
      § | 
    
  | 11:34 | <marostegui> | Test snapshot db1117:3325 to dbstore1001 - T210292 | [production] | 
            
  | 10:55 | <marostegui> | Upgrade db2057 | [production] | 
            
  | 10:00 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1085 (duration: 00m 56s) | [production] | 
            
  | 09:52 | <mutante> | ms-be1035 - sudo  systemctl reset-failed | [production] | 
            
  | 09:45 | <ema> | cp1071: upgrade trafficserver to 8.0.3~rc0 for testing purposes | [production] | 
            
  | 09:41 | <marostegui> | Deploy schema change on db1085 with replication, there will be lag on labsdb:s6 | [production] | 
            
  | 09:41 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1085 (duration: 00m 55s) | [production] | 
            
  | 09:06 | <moritzm> | installing PHP 7.0 security updates | [production] | 
            
  | 08:59 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1113:3316 (duration: 00m 55s) | [production] | 
            
  | 08:58 | <marostegui> | Upgrade mysql and kernel on db2050 | [production] | 
            
  | 08:51 | <ema> | cp3030: wipe frontend cache to get rid of large objects T216006 | [production] | 
            
  | 08:26 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1113:3316 (duration: 00m 55s) | [production] | 
            
  | 08:14 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1093 (duration: 00m 55s) | [production] | 
            
  | 08:09 | <moritzm> | upgrading job runners in eqiad to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) | [production] | 
            
  | 07:30 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1093 (duration: 00m 54s) | [production] | 
            
  | 07:26 | <moritzm> | upgrading remaining app servers in eqiad to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) | [production] | 
            
  | 07:25 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Fully repool db1096 (duration: 00m 58s) | [production] | 
            
  | 07:13 | <marostegui> | Test snapshot dbstore1001:3311 to dbstore1001 - T210292 | [production] | 
            
  | 07:12 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: More traffic to db1096 (duration: 00m 55s) | [production] | 
            
  | 06:58 | <marostegui> | Upgrade MySQL and kernel on db2036 | [production] | 
            
  | 06:56 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Slowly repool db1096 (duration: 00m 55s) | [production] | 
            
  | 06:40 | <marostegui> | Stop MySQL on db1096 for upgrade | [production] | 
            
  | 06:24 | <kart_> | Finished manual run of unpublished ContentTranslation draft purge script (T217818) | [production] | 
            
  | 06:21 | <marostegui> | Testing snapshotting on db1117:3321 to > dbstore1001 - T210292 | [production] | 
            
  | 06:15 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1096 (duration: 01m 07s) | [production] | 
            
  | 04:11 | <kart_> | Started manual run of unpublished ContentTranslation draft purge script (T217818) | [production] | 
            
  
    | 2019-03-12
      
      § | 
    
  | 23:33 | <thcipriani@deploy1001> | Synchronized php-1.33.0-wmf.21/extensions/MobileFrontend/includes/specials/SpecialMobileOptions.php: SWAT: [[gerrit:495907|Fix: undefined locals in SpecialMobileOptions.setJsConfigVars()]] T218098 (duration: 00m 57s) | [production] | 
            
  | 20:49 | <shdubsh> | manually upgrade prometheus-icinga-exporter to 0.5 on standby icinga | [production] | 
            
  | 19:48 | <eileen> | civicrm revision changed from 977b9bfcf1 to 5c45e4c24d, config revision is f930677e97 | [production] | 
            
  | 19:31 | <herron> | restarted citoid on scb1003 | [production] | 
            
  | 19:16 | <otto@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Enabling api-request logging to eventgate-analytics for group0 wikis (duration: 01m 01s) | [production] | 
            
  | 19:14 | <arturo> | T216497 manually delete libpam-systemd and libsystemd0 230-7~bpo8+2 from jessie-wikimedia/openstack-mitaka-jessie | [production] | 
            
  | 19:09 | <arturo> | T216497 manually delete systemd 230-7~bpo8+2 from jessie-wikimedia/openstack-mitaka-jessie | [production] | 
            
  | 19:07 | <robh> | rebooting thumbor1004 for memory troubleshooting via T215411 | [production] | 
            
  | 17:11 | <addshore@deploy1001> | Synchronized php-1.33.0-wmf.21/extensions/Wikibase/repo/includes/Store/Sql/SqlStore.php: T97368 Increase APC cache for PropertyInfoLookup from 15 to 20s (duration: 00m 55s) | [production] | 
            
  | 17:10 | <addshore@deploy1001> | Synchronized php-1.33.0-wmf.20/extensions/Wikibase/repo/includes/Store/Sql/SqlStore.php: T97368 Increase APC cache for PropertyInfoLookup from 15 to 20s (duration: 00m 57s) | [production] | 
            
  | 17:02 | <jbond42> | rolling update of debdeploy | [production] | 
            
  | 16:57 | <addshore@deploy1001> | Synchronized wmf-config/InitialiseSettings-labs.php: beta only (duration: 00m 53s) | [production] | 
            
  | 16:43 | <addshore@deploy1001> | Synchronized php-1.33.0-wmf.21/extensions/Wikibase/repo/includes/Store/Sql/SqlStore.php: T97368 Double on server cache for PropertyInfoStore (duration: 00m 55s) | [production] | 
            
  | 16:42 | <addshore@deploy1001> | Synchronized php-1.33.0-wmf.20/extensions/Wikibase/repo/includes/Store/Sql/SqlStore.php: T97368 Double on server cache for PropertyInfoStore (duration: 00m 57s) | [production] | 
            
  | 16:29 | <moritzm> | upgraded buster installation image to daily build from 12th of March (T213527) | [production] | 
            
  | 15:45 | <otto@deploy1001> | scap-helm eventgate-analytics cluster codfw completed | [production] | 
            
  | 15:45 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 15:45 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade production -f eventgate-analytics-codfw-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: codfw] | [production] | 
            
  | 15:43 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 15:43 | <otto@deploy1001> | scap-helm eventgate-analytics cluster eqiad completed | [production] | 
            
  | 15:43 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade production -f eventgate-analytics-eqiad-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: eqiad] | [production] | 
            
  | 15:43 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade production -f eventgate-analytics-eqiad-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: eqiad] | [production] | 
            
  | 15:42 | <ppchelko@deploy1001> | scap-helm eventgate-analytics upgrade production -f eventgate-analytics-eqiad-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: eqiad] | [production] | 
            
  | 15:41 | <ppchelko@deploy1001> | scap-helm eventgate-analytics upgrade -f eventgate-analytics-eqiad-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: eqiad] | [production] |