tag:status.etherfax.net,2005:/historyetherFAX Status - Incident History2024-03-29T01:22:37-04:00etherFAXtag:status.etherfax.net,2005:Incident/189757912023-10-31T07:26:08-04:002023-10-31T07:26:08-04:00DE1 - ISP Outage Affecting Site’s Connectivity<p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>07:26</var> EDT</small><br><strong>Resolved</strong> - Our connectivity partner has confirmed this outage has been resolved.</p><p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>02:17</var> EDT</small><br><strong>Identified</strong> - Our connectivity partner in DE1 (Denver, CO) is experiencing and currently working to resolve an outage in the region. Site redundancy is at a reduced level during this outage.</p>tag:status.etherfax.net,2005:Incident/181433752023-09-14T14:07:27-04:002023-09-14T14:07:27-04:00EtherFAX Portal SMS code delays<p><small>Sep <var data-var='date'>14</var>, <var data-var='time'>14:07</var> EDT</small><br><strong>Resolved</strong> - etherFAX has begun utilizing a new from number for SMS delivery. SMS delivery should now complete successfully. <br /><br />As noted previously, future security compliance will eventually disallow SMS as a secure method and there will be forthcoming announcements to that effect. If you are able to log into the portal with an SMS code today, you can switch your 2FA method over now by clicking your Username in the bottom left nav, then Manage Two-Factor. Click Set Up Now under the Google Authenticator section once ensuring you have an associated compatible app to pair the QR code on your phone. <br /><br />Google Authenticator and Microsoft Authenticator are known compatible, but there may be more TOTP-compatible apps available for your use. Check with your security team if needed for approved multi-factor applications.</p><p><small>Aug <var data-var='date'>14</var>, <var data-var='time'>13:10</var> EDT</small><br><strong>Update</strong> - Adding a link to our KB for configuring 2FA.<br />https://kb.etherfax.net/cportal/how-to-set-up-change-or-reset-your-two-factor-auth</p><p><small>Aug <var data-var='date'>14</var>, <var data-var='time'>12:57</var> EDT</small><br><strong>Monitoring</strong> - EtherFax engineers have been made aware of sporadic and carrier-specific delays when receiving portal login 2FA codes via SMS. We are actively engaged with SMS providers on the issue, but have learned that there are additional obstacles imposed at the carrier level that have pushed a possible ETA for resolution out substantially. <br /><br />For anyone not receiving your 2FA portal login codes via SMS, we recommend switching to Authenticator-based (Google/Microsoft Authenticator, etc) tokens. You can do this by reaching out to support@etherfax.net, referencing this issue.<br /><br />In addition, we would highlight this as an opportunity to move your 2FA over to an Authenticator method. Future security compliance will eventually disallow SMS as a secure method and there will be forthcoming announcements to that effect. If you are able to log into the portal with an SMS code today, you can switch your 2FA method over now by clicking your Username in the bottom left nav, then Manage Two-Factor. Click Set Up Now under the Google Authenticator section once ensuring you have an associated compatible app to pair the QR code on your phone. <br /><br />Google Authenticator and Microsoft Authenticator are known compatible, but there may be more TOTP-compatible apps available for your use. Check with your security team if needed for approved multi-factor applications.</p>tag:status.etherfax.net,2005:Incident/171092852023-05-01T11:43:14-04:002023-05-01T11:43:14-04:00etherFAX Portals Experiencing Degraded Performance<p><small>May <var data-var='date'> 1</var>, <var data-var='time'>11:43</var> EDT</small><br><strong>Resolved</strong> - Performance has remained a normal levels. This incident is now closed.</p><p><small>May <var data-var='date'> 1</var>, <var data-var='time'>10:15</var> EDT</small><br><strong>Monitoring</strong> - Engineers have addressed the issue and logging and monitoring metrics show portal performance is back to normal levels. We will continue to monitor.</p><p><small>May <var data-var='date'> 1</var>, <var data-var='time'>09:19</var> EDT</small><br><strong>Update</strong> - Database engineers have been engaged to address what is believed to be the root cause for the sluggishness. Updates will continue to be posted here as they are available.</p><p><small>May <var data-var='date'> 1</var>, <var data-var='time'>09:04</var> EDT</small><br><strong>Investigating</strong> - Engineers are currently investigating degraded performance on the etherFAX portals.</p>tag:status.etherfax.net,2005:Incident/165285872023-03-25T00:00:07-04:002023-03-25T00:00:07-04:00[NY1] New York ISP Maintenance<p><small>Mar <var data-var='date'>25</var>, <var data-var='time'>00:00</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>24</var>, <var data-var='time'>20:00</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>09:47</var> EDT</small><br><strong>Scheduled</strong> - One of our ISPs in the NY1 data center has alerted etherFAX to maintenance that will occur during this window. We will be adjusting DNS records to point away from this ISP circuit. There is no expected impact to customers during this time.</p>tag:status.etherfax.net,2005:Incident/136616302022-11-23T13:16:00-05:002022-11-23T13:16:00-05:00[TR1] - Outbound Call Failures<p><small>Nov <var data-var='date'>23</var>, <var data-var='time'>13:16</var> EST</small><br><strong>Resolved</strong> - At this time carrier and colocation provider have determined there was damage to cabling entering the data center due to construction. This is under repair currently but they have routed traffic around affected infrastructure.</p><p><small>Nov <var data-var='date'>23</var>, <var data-var='time'>12:04</var> EST</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Nov <var data-var='date'>23</var>, <var data-var='time'>12:04</var> EST</small><br><strong>Monitoring</strong> - Carrier has verified a circuit was in alarm on their side. They have corrected this and etherFAX engineers are continuing to monitor before returning traffic to the normal paths.</p><p><small>Nov <var data-var='date'>23</var>, <var data-var='time'>11:43</var> EST</small><br><strong>Identified</strong> - Engineers have verified that a data center cross connect is down between etherFAX and our carrier. Data center and carrier engineers have been engaged and are troubleshooting the outage. At this time we have implemented a workaround for outbound calls and those calls should be completing.</p><p><small>Nov <var data-var='date'>23</var>, <var data-var='time'>11:12</var> EST</small><br><strong>Investigating</strong> - Engineers are currently engaged with outbound carrier in our TR1 facility and actively troubleshooting outbound call failures.</p>tag:status.etherfax.net,2005:Incident/113017832022-10-05T20:56:09-04:002022-10-05T20:56:09-04:00Hurricane Ian<p><small>Oct <var data-var='date'> 5</var>, <var data-var='time'>20:56</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'>28</var>, <var data-var='time'>12:42</var> EDT</small><br><strong>Monitoring</strong> - The etherFAX team is closely watching impacts felt from Hurricane Ian. First and foremost please be safe if you are in the path. If you need support from the etherFAX team during or in the aftermath of the storm please reach out to our team. We will be closely monitoring any customer tickets 24x7 which have sites/services impacted due to the storm. No impact to etherFAX infrastructure is expected.</p>tag:status.etherfax.net,2005:Incident/98513742022-04-25T23:29:12-04:002022-04-25T23:29:12-04:00US-NY1<p><small>Apr <var data-var='date'>25</var>, <var data-var='time'>23:29</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'>25</var>, <var data-var='time'>14:27</var> EDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Apr <var data-var='date'>25</var>, <var data-var='time'>14:27</var> EDT</small><br><strong>Update</strong> - The issue has stabilized and engineers from both carrier and etherFAX sides are seeing good call completions. Root cause determination is on-going.</p><p><small>Apr <var data-var='date'>25</var>, <var data-var='time'>13:16</var> EDT</small><br><strong>Update</strong> - Engineers are working with the carrier to help isolate this issue.</p><p><small>Apr <var data-var='date'>25</var>, <var data-var='time'>12:50</var> EDT</small><br><strong>Investigating</strong> - Engineers are currently investigating reports of inbound toll calls failing to connect in our NY data center.</p>tag:status.etherfax.net,2005:Incident/90219082022-01-07T00:02:29-05:002022-01-07T00:02:29-05:00[US-California] Inbound DID Issue Affecting DID's in California Region<p><small>Jan <var data-var='date'> 7</var>, <var data-var='time'>00:02</var> EST</small><br><strong>Resolved</strong> - Carrier engineering and hardware team found an ingress switch card which was in an error status. They have removed this from service and are routing calls across redundant hardware. etherFAX engineers are showing the error clearing after the action by the carrier team. We will continue to monitor.</p><p><small>Jan <var data-var='date'> 6</var>, <var data-var='time'>22:32</var> EST</small><br><strong>Update</strong> - The etherFAX team is still engaged with the carrier team during their escalation. We are seeing this affect other DIDs outside of the California area.</p><p><small>Jan <var data-var='date'> 6</var>, <var data-var='time'>21:23</var> EST</small><br><strong>Investigating</strong> - etherFAX Engineers are currently engaged with our carrier regarding a regional issue impacting inbound fax calls from certain sending carriers failing to complete to DIDs in the California region. The carrier's engineering team is escalating internally.</p>tag:status.etherfax.net,2005:Incident/87836032021-12-12T18:01:33-05:002021-12-12T18:01:33-05:00[US] Inbound Toll DID Call Failures<p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>18:01</var> EST</small><br><strong>Resolved</strong> - This incident is being closed out of monitoring status.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>22:04</var> EST</small><br><strong>Update</strong> - All impacted call paths remain operational. Engineers will monitor call progress overnight as a precaution.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>14:45</var> EST</small><br><strong>Update</strong> - Call paths are operational and engineers continue to monitor with the carrier.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>14:31</var> EST</small><br><strong>Monitoring</strong> - Workarounds are in place for all numbers and they should be ringing to etherFAX as expected. Engineers will continue to monitor call completion.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>13:59</var> EST</small><br><strong>Update</strong> - Partial restoration is being seen as numbers begin to ring again. Engineers are continuing to work on the larger issue and provide alternative call paths in tandem.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>13:34</var> EST</small><br><strong>Update</strong> - Engineers remain engaged with carriers on this issue, and are implementing workarounds to land calls in the interim.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>12:37</var> EST</small><br><strong>Investigating</strong> - etherFAX engineers are investigating degraded inbound calls affecting one of our underlying carriers. etherFAX is engaged with the underlying carrier engineering group and are actively troubleshooting.</p>tag:status.etherfax.net,2005:Incident/87159132021-12-09T01:00:31-05:002021-12-09T01:00:31-05:00[NY1] Network Infrastructure Maintenance<p><small>Dec <var data-var='date'> 9</var>, <var data-var='time'>01:00</var> EST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Dec <var data-var='date'> 8</var>, <var data-var='time'>22:00</var> EST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Nov <var data-var='date'>29</var>, <var data-var='time'>09:53</var> EST</small><br><strong>Scheduled</strong> - etherFAX Network Engineers will be performing routine maintenance on network hardware. During this time network traffic may be routed over alternate ISP's to avoid any service disruption.</p>tag:status.etherfax.net,2005:Incident/87277272021-11-30T17:56:59-05:002021-11-30T17:56:59-05:00[US] Upstream Carrier Issues<p><small>Nov <var data-var='date'>30</var>, <var data-var='time'>17:56</var> EST</small><br><strong>Resolved</strong> - We are continuing to observe normal call completions from the affected upstream carrier. This incident is now resolved.</p><p><small>Nov <var data-var='date'>30</var>, <var data-var='time'>17:22</var> EST</small><br><strong>Monitoring</strong> - The upstream carrier has resolved the issue as of 22:15 UTC and we are continuing to monitor.</p><p><small>Nov <var data-var='date'>30</var>, <var data-var='time'>16:54</var> EST</small><br><strong>Investigating</strong> - etherFAX is aware of an upstream carrier issue affecting some toll DIDs on our network. Engineers are monitoring and in touch with the upstream carrier engineers.</p>tag:status.etherfax.net,2005:Incident/83274512021-10-27T00:52:34-04:002021-10-27T00:52:34-04:00API Connection Failures<p><small>Oct <var data-var='date'>27</var>, <var data-var='time'>00:52</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'>26</var>, <var data-var='time'>23:38</var> EDT</small><br><strong>Monitoring</strong> - Critical emergency maintenance has been completed and engineers are continuing to monitor the components.</p><p><small>Oct <var data-var='date'>26</var>, <var data-var='time'>23:32</var> EDT</small><br><strong>Identified</strong> - Engineers will be performing a emergency database maintenance to assist in mitigating the issue observed. Some connection issues may be observed over the next few minutes.</p><p><small>Oct <var data-var='date'>26</var>, <var data-var='time'>23:30</var> EDT</small><br><strong>Update</strong> - Engineers are continuing to monitor the issue. Continued updates to follow.</p><p><small>Oct <var data-var='date'>26</var>, <var data-var='time'>22:50</var> EDT</small><br><strong>Investigating</strong> - Engineers are currently investigating some connection failures and delays connecting to the etherFAX API.</p>tag:status.etherfax.net,2005:Incident/82208362021-10-14T12:26:28-04:002021-10-14T12:26:28-04:00Inbound DID from some areas in US impacted<p><small>Oct <var data-var='date'>14</var>, <var data-var='time'>12:26</var> EDT</small><br><strong>Resolved</strong> - After monitoring call completion rates overnight, engineers feel confident that this issue has been resolved and that calls are completing as expected.</p><p><small>Oct <var data-var='date'>13</var>, <var data-var='time'>18:43</var> EDT</small><br><strong>Monitoring</strong> - A routing change has been put into place to address this issue. Calls over the impacted carrier appear to be completing as expected and engineers continue to monitor traffic.</p><p><small>Oct <var data-var='date'>13</var>, <var data-var='time'>17:15</var> EDT</small><br><strong>Identified</strong> - Engineers remain engaged with the carrier on this issue, and have isolated the issue to a change made within the carrier interconnects.</p><p><small>Oct <var data-var='date'>13</var>, <var data-var='time'>13:10</var> EDT</small><br><strong>Investigating</strong> - EtherFAX engineers are aware of an issue with select inbound DID footprint and are actively engaged with carriers on investigating the matter.</p>tag:status.etherfax.net,2005:Incident/76364892021-08-21T04:03:17-04:002021-08-21T04:03:17-04:00New York (NY1) ISP Maintenance<p><small>Aug <var data-var='date'>21</var>, <var data-var='time'>04:03</var> EDT</small><br><strong>Completed</strong> - The ISP has just advised that the maintenance has been postponed and will not occur during this window due to some equipment not being ready. They will reschedule this window in the future.</p><p><small>Aug <var data-var='date'>21</var>, <var data-var='time'>03:00</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Aug <var data-var='date'> 3</var>, <var data-var='time'>16:50</var> EDT</small><br><strong>Update</strong> - The ISP has updated this maintenance to 8/21/2021.</p><p><small>Jul <var data-var='date'>29</var>, <var data-var='time'>15:45</var> EDT</small><br><strong>Scheduled</strong> - Our primary ISP in the NY1 data center has re-scheduled the previously postponed maintenance window on equipment servicing our circuits. During the newly scheduled window etherFAX will failover to an alternate ISP. No impact to customer functionality is expected. They have specified a 3 hour window although the maintenance itself is only expected to take 1 hour.</p>tag:status.etherfax.net,2005:Incident/76064262021-08-08T06:00:30-04:002021-08-08T06:00:30-04:00[TR1/Toronto] - ISP Scheduled Maintenance<p><small>Aug <var data-var='date'> 8</var>, <var data-var='time'>06:00</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Aug <var data-var='date'> 8</var>, <var data-var='time'>00:00</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jul <var data-var='date'>26</var>, <var data-var='time'>10:10</var> EDT</small><br><strong>Scheduled</strong> - Our ISP in the Toronto data center will be performing scheduled maintenance on equipment servicing our circuits. etherFAX will failover to an alternate ISP. No impact to customer functionality is expected. They have specified a 6 hour window although the maintenance itself is only expected to take 1 hour.</p>tag:status.etherfax.net,2005:Incident/74568602021-08-01T11:53:05-04:002021-08-01T11:53:05-04:00SSL Certificate Renewal & Update<p><small>Aug <var data-var='date'> 1</var>, <var data-var='time'>11:53</var> EDT</small><br><strong>Completed</strong> - The certificate update has been completed successfully. This maintenance window is now closed.</p><p><small>Aug <var data-var='date'> 1</var>, <var data-var='time'>11:00</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jul <var data-var='date'> 9</var>, <var data-var='time'>12:30</var> EDT</small><br><strong>Scheduled</strong> - On August 1st, 2021 at 11am EDT etherFAX will be updating our SSL certificate used for API services for the na.connect.etherfax.net, connect.etherfax.net, and ca.connect.etherfax.net endpoints. This is an advisory notice as no impact to customers is expected.</p>tag:status.etherfax.net,2005:Incident/75665242021-07-24T23:34:41-04:002021-07-24T23:34:41-04:00Database Replication Maintenance<p><small>Jul <var data-var='date'>24</var>, <var data-var='time'>23:34</var> EDT</small><br><strong>Completed</strong> - Engineers have completed work for this window. All services are operational.</p><p><small>Jul <var data-var='date'>24</var>, <var data-var='time'>21:01</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jul <var data-var='date'>21</var>, <var data-var='time'>09:46</var> EDT</small><br><strong>Scheduled</strong> - etherFAX Database Administrators will be performing database replication maintenance during this time. No impact to services are expected.</p>tag:status.etherfax.net,2005:Incident/71779042021-06-07T07:55:47-04:002021-06-07T07:55:47-04:00Continuation of Database Maintenance Window<p><small>Jun <var data-var='date'> 7</var>, <var data-var='time'>07:55</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>17:12</var> EDT</small><br><strong>Verifying</strong> - Maintenance window has concluded. All systems are functioning normally. Engineers will continue to monitor the system closely over the next few hours.</p><p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>12:48</var> EDT</small><br><strong>In progress</strong> - Engineers have resumed maintenance work on the database infrastructure, specifically around the replication components between sites. Data centers continue to operate normally on an individual basis.</p><p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>06:04</var> EDT</small><br><strong>Update</strong> - Engineers are pausing work on the site replication portions of the maintenance. All data centers continue to operate independently and work will resume in several hours on the replication components.</p><p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>04:42</var> EDT</small><br><strong>Update</strong> - All data centers continue to function. Engineers are still working on completing replication verification between sites.</p><p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>02:58</var> EDT</small><br><strong>Verifying</strong> - Engineers are continuing to work on the upgrade verification. Traffic continues to flow normally.</p><p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>00:07</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>00:05</var> EDT</small><br><strong>Scheduled</strong> - etherFAX Engineers continue to verify the database hardware maintenance.</p>tag:status.etherfax.net,2005:Incident/69317032021-06-06T02:00:29-04:002021-06-06T00:01:39-04:00[NY1] - Database Hardware Upgrade<p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>02:00</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jun <var data-var='date'> 5</var>, <var data-var='time'>20:00</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>May <var data-var='date'> 5</var>, <var data-var='time'>22:44</var> EDT</small><br><strong>Scheduled</strong> - etherFAX engineers will be performing a hardware upgrade on the NY1 database infrastructure. During this time system loads and traffic will be transferred to our Denver, US data center as needed. No customer interactions is required. There may be momentary web service timeouts as the cutover to new hardware occurs.</p>tag:status.etherfax.net,2005:Incident/68548192021-04-27T14:34:17-04:002021-04-27T14:34:17-04:00[TR1] Canadian Telephony Issue<p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>14:34</var> EDT</small><br><strong>Resolved</strong> - The carrier advises the issue is resolved and their team is still investigating the root cause and will publish an RFO in the coming days. We will pass along that detail once we have it.</p><p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>12:53</var> EDT</small><br><strong>Monitoring</strong> - The carrier has advised they are starting to gradually process calls. Their engineering and operations team continues to monitor.</p><p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>11:56</var> EDT</small><br><strong>Identified</strong> - etherFAX engineers are going to repoint the Canadian service URL to our US data center temporarily in order to service outbound faxes. Inbound faxing will still remain affected.</p><p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>11:51</var> EDT</small><br><strong>Investigating</strong> - Our Canadian telephony carrier is currently experiencing a network wide disruption to their services. We are engaged with them and awaiting updates.</p>tag:status.etherfax.net,2005:Incident/65013252021-03-10T20:39:01-05:002021-03-17T10:38:30-04:00DNS Resolution Failures<p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>20:39</var> EST</small><br><strong>Resolved</strong> - DNS provider advises that mitigating actions have been taken and services are recovered and stable. Further information to follow as well as a post-mortem for etherFAX will be provided.</p><p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>20:07</var> EST</small><br><strong>Update</strong> - Engineers did add additional nameservers for an alternate provider as well. DNS logging may reflect alternate name servers. Resolution has continued to remain stable.</p><p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>19:43</var> EST</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>19:40</var> EST</small><br><strong>Monitoring</strong> - We're seeing resolution and it appears the DNS provider is back up. We're continuing to monitor.</p><p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>19:12</var> EST</small><br><strong>Update</strong> - We are currently engaged with our DNS provider to determine extent of outage and an ETR. A host file can be temporarily added for the following based on your connection point:<br /><br />US: na.connect.etherfax.net 4.71.144.154<br />connect.etherfax.net 4.71.144.154<br />CA: ca.connect.etherfax.net 4.28.142.21</p><p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>18:51</var> EST</small><br><strong>Investigating</strong> - Engineers are currently monitoring reports of DNS resolution failures with our DNS provider. More updates to follow.</p>tag:status.etherfax.net,2005:Incident/60055842021-02-06T21:04:23-05:002021-02-06T21:04:23-05:00TR1 Database Maintenance and Upgrades<p><small>Feb <var data-var='date'> 6</var>, <var data-var='time'>21:04</var> EST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'> 6</var>, <var data-var='time'>20:54</var> EST</small><br><strong>Verifying</strong> - Maintenance has completed and services are restored. Traffic in the Toronto (TR1) datacenter has returned to normal. Engineers are performing final validation of services.</p><p><small>Feb <var data-var='date'> 6</var>, <var data-var='time'>16:00</var> EST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jan <var data-var='date'>12</var>, <var data-var='time'>09:52</var> EST</small><br><strong>Scheduled</strong> - etherFAX engineers and database administrators will be performing database maintenance and upgrades to the Toronto (TR1) database infrastructure. During this time customers who are connecting to the Canadian service URL will have their traffic will be moved off of this instance using a DNS change. No impact to customer service is expected.</p>tag:status.etherfax.net,2005:Incident/59613182021-01-14T06:00:52-05:002021-01-14T06:00:52-05:00Toronto (TR1) ISP Maintenance<p><small>Jan <var data-var='date'>14</var>, <var data-var='time'>06:00</var> EST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jan <var data-var='date'>14</var>, <var data-var='time'>00:01</var> EST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jan <var data-var='date'> 6</var>, <var data-var='time'>09:47</var> EST</small><br><strong>Scheduled</strong> - Our ISP in the Toronto data center will be performing scheduled maintenance on equipment servicing our circuits. etherFAX will failover to an alternate ISP. No impact to customer functionality is expected. They have specified a 6 hour window although the maintenance itself is only expected to take 1 hour.</p>tag:status.etherfax.net,2005:Incident/55811152020-11-16T20:33:47-05:002020-11-16T20:33:47-05:00DE1 ISP Issues<p><small>Nov <var data-var='date'>16</var>, <var data-var='time'>20:33</var> EST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Nov <var data-var='date'>16</var>, <var data-var='time'>16:46</var> EST</small><br><strong>Monitoring</strong> - Engineers have continued to see positive ping responses and stability on the circuit. This was part of a bigger outage with the ISP and we are waiting for more updates from them.</p><p><small>Nov <var data-var='date'>16</var>, <var data-var='time'>16:32</var> EST</small><br><strong>Update</strong> - Packet loss and stability have returned to normal levels.</p><p><small>Nov <var data-var='date'>16</var>, <var data-var='time'>16:13</var> EST</small><br><strong>Investigating</strong> - Engineers are currently monitoring a service provider issue affecting our DE1 (Denver, CO) data center. There should be no impact to services at this time.</p>tag:status.etherfax.net,2005:Incident/49737772020-08-30T14:11:18-04:002020-08-30T14:11:18-04:00Monitoring Broad Internet Issues across US<p><small>Aug <var data-var='date'>30</var>, <var data-var='time'>14:11</var> EDT</small><br><strong>Resolved</strong> - etherFAX Engineers have just updated our service address DNS record back to our primary ISP. All systems have been operating normally and with no alarms.</p><p><small>Aug <var data-var='date'>30</var>, <var data-var='time'>11:42</var> EDT</small><br><strong>Update</strong> - The provider has made a configuration change and is starting to see alarms clear. etherFAX is also seeing indications of this in our external monitoring.</p><p><small>Aug <var data-var='date'>30</var>, <var data-var='time'>11:02</var> EDT</small><br><strong>Update</strong> - We've received this update from CenturyLink as part of their master ticket for this issue: "The IP NOC has isolated multiple Border Gateway Protocol (BGP) issues causing service impacts across multiple markets. Cooperative escalated investigation efforts with leadership are underway to isolate and troubleshoot the issue and expedite service restoral."</p><p><small>Aug <var data-var='date'>30</var>, <var data-var='time'>10:07</var> EDT</small><br><strong>Monitoring</strong> - Engineers continue to monitor reports on this incident from the major providers. As stated previously, etherFAX services are up and functioning. We will continue to receive your inbound faxes and they will be available once you re-establish connection. If you are having connection issues reaching us it is extremely likely they are caused by the ongoing IP routing issue. We'll continue to provide any updates we receive on this.</p><p><small>Aug <var data-var='date'>30</var>, <var data-var='time'>08:49</var> EDT</small><br><strong>Update</strong> - Based on further reporting as to the possible causes of this global routing issue engineers updated our DNS record off of our Primary ISP and over to our secondary ISP. This was done to further mitigate any routing issues customers were experiencing reaching our services. We are monitoring customer connection counts and have seen considerable improvement after this update which effectively re-routes how customers (and their ISPs) attempt to reach us.</p><p><small>Aug <var data-var='date'>30</var>, <var data-var='time'>08:15</var> EDT</small><br><strong>Update</strong> - Updates from major providers indicates a possible global routing issue. Engineers are continuing to monitor the situation.</p><p><small>Aug <var data-var='date'>30</var>, <var data-var='time'>06:55</var> EDT</small><br><strong>Update</strong> - Our Canadian data center and service URL is being affected by this outage. While this does appear to be a broader issues we have tickets open with our providers for tracking and informational purposes.</p><p><small>Aug <var data-var='date'>30</var>, <var data-var='time'>06:54</var> EDT</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Aug <var data-var='date'>30</var>, <var data-var='time'>06:40</var> EDT</small><br><strong>Investigating</strong> - Engineers were alerted to external monitoring checks failing from certain locations around the US. We're seeing many reports of other major services also being affected. We will continue to monitor and provide any updates as we receive them. We have verified that the etherFAX system itself is functioning and responding to requests.</p>