Untitled

Meeting Call Overview

• Scheduled Call: 11:30 PM Eastern time

• Attendees: Sai, Palo Alto representatives

• Purpose: Deeper analysis of ongoing issues

Technical Requests and iPerf Testing

• Request to Sai to download iPerf

  • iPerf purpose: Network testing utility to measure bandwidth performance between two computers

  • Interest in testing: UDP vs TCP

  • Target metrics: Megabytes per second

• Availability of iPerf depends on the laptop size and permissions

  • Sai's possible restrictions on administrative rights to install software

  • Alternative: Request installation through service desk if needed

Preparations for Upcoming Calls

• Reminder to contact Sai regarding iPerf installation

  • Suggestion to send direct message or email for awareness

  • Confirmation of call timings in his local timezone (10 AM IST)

February Schedule Discussions

• Confirmed testing events scheduled in February

  • Budapest event scheduled for today

  • Chandler and Ashburn starting at 8 AM on February 7

  • Discussion needed about pushing the schedule due to migration concerns

  • Ensure communication with the director for proper notifications

• VPN Testing scheduled in Provo and Newberry on February 10 at 9 PM

• Milan, Italy scheduled for February 13 with confirmations received

• Germany scheduled discussions confirmed for February 12 and 13

Site Issues and Incident Discussions

• Lithia Springs site concerns

  • Initial troubles, but upgrade completed; current status: stable

  • No open incidents reported regarding Lithia Springs

• Follow-up on incident tracking and device return

  • Device cleanup ongoing, with involvement from Brad and Marissa

• GCP (Google Cloud Platform) virtual ions project mentioned, focusing on IPv6 design testing

Incident Priority Levels and Site Classification

• Discussion on priority levels assigned to incidents

  • Proposal to evaluate proper categorization of critical sites as P1

  • Reminder of existing site classification hierarchy for incident reporting:

  • P1: Critical incidents (such as data center outages)

  • P2: Site down, but not critical (secondary branches)

• Query about the status of alerts and categorizations in alerts

  • Emphasis on clarity of response actions based on priority levels

  • Definition needed for BGP (Border Gateway Protocol) issue categorization as P1 or P2

• The point raised that having a site down should ideally indicate a P1 situation due to user impact

Review and Communication on Alerts and Service Levels

• Proposal for better documentation and understanding of priorities based on criticality

  • Discussion on response escalations and how alerts are categorized correctly

  • Potential creation of a flowchart for response actions based on alert types

Feedback and Wrap-Up

• The necessity for better tracking of SLAs (Service Level Agreements) for alerts and responses

• Agreement to continue discussions in future meetings to align priorities and ensure effective resolutions across technical teams

Based on the notes provided, there is no mention of ignoring the Chandler and Ashburn sites. The specific record states that they were scheduled to start on February $7$ at $8$ AM, but there was a discussion regarding pushing the schedule (postponing) because of migration concerns. It was also noted that communication with the Director is required for proper notification, but the sites themselves remain part of the project scope.

Meeting Call Overview

• Scheduled Call: $11:30$ PM Eastern time

• Attendees: Sai, Palo Alto representatives

• Purpose: Deeper analysis of ongoing issues

Technical Requests and iPerf Testing

• Request to Sai to download iPerf

  • iPerf purpose: Network testing utility to measure bandwidth performance between two computers

  • Interest in testing: $UDP$ vs $TCP$

  • Target metrics: Megabytes per second ($MB/s$)

• Availability of iPerf depends on the laptop size and permissions

  • Sai's possible restrictions on administrative rights to install software

  • Alternative: Request installation through service desk if needed

Preparations for Upcoming Calls

• Reminder to contact Sai regarding iPerf installation

  • Suggestion to send direct message or email for awareness

  • Confirmation of call timings in his local timezone ($10$ AM IST)

February Schedule Discussions

• Confirmed testing events scheduled in February

  • Budapest event scheduled for today

  • Chandler and Ashburn starting at $8$ AM on February $7$

  • Discussion needed about pushing the schedule due to migration concerns

  • Ensure communication with the director for proper notifications

• VPN Testing scheduled in Provo and Newberry on February $10$ at $9$ PM

• Milan, Italy scheduled for February $13$ with confirmations received

• Germany scheduled discussions confirmed for February $12$ and $13$

Site Issues and Incident Discussions

• Lithia Springs site concerns

  • Initial troubles, but upgrade completed; current status: stable

  • No open incidents reported regarding Lithia Springs

• Follow-up on incident tracking and device return

  • Device cleanup ongoing, with involvement from Brad and Marissa

• GCP (Google Cloud Platform) virtual ions project mentioned, focusing on $IPv6$ design testing

Incident Priority Levels and Site Classification

• Discussion on priority levels assigned to incidents

  • Proposal to evaluate proper categorization of critical sites as $P1$

  • Reminder of existing site classification hierarchy for incident reporting:

  • $P1$: Critical incidents (such as data center outages)

  • $P2$: Site down, but not critical (secondary branches)

• Query about the status of alerts and categorizations in alerts

  • Emphasis on clarity of response actions based on priority levels

  • Definition needed for $BGP$ (Border Gateway Protocol) issue categorization as $P1$ or $P2$

• The point raised that having a site down should ideally indicate a $P1$ situation due to user impact

Review and Communication on Alerts and Service Levels

• Proposal for better documentation and understanding of priorities based on criticality

  • Discussion on response escalations and how alerts are categorized correctly

  • Potential creation of a flowchart for response actions based on alert types

Feedback and Wrap-Up

• The necessity for better tracking of SLAs (Service Level Agreements) for alerts and responses

• Agreement to continue discussions in future meetings to align priorities and ensure effective resolutions across technical teams

Based on the notes provided, there is no mention of ignoring the Chandler and Ashburn sites. The specific record states that they were scheduled to start on February $7$ at $8$ AM, but there was a discussion regarding pushing the schedule (postponing) because of migration concerns. It was also noted that communication with the Director is required for proper notification, but the sites themselves remain part of the project scope.

Detailed Summary of Strategic Objectives

1. Meeting Call Overview and Strategic Objectives

   • Scheduled Session: The technical review is set for $11:30$ PM Eastern Time.

   • Participants: Primary engineer Sai, accompanied by representatives from Palo Alto Networks.

   • Primary Purpose: Conduct a comprehensive deep-dive into ongoing performance bottlenecks and synchronize on the long-term migration roadmap.

2. Network Performance Testing with iPerf

   • Deployment Request: Sai is tasked with the installation and configuration of the iPerf utility.

   • iPerf Technical Function: A cross-platform tool used for active measurements of the maximum achievable bandwidth on IP networks. It works on a client-server model to generate traffic.

   • Protocol Comparison: The team is specifically interested in comparing $UDP$ (User Datagram Protocol) and $TCP$ (Transmission Control Protocol) performance.

     • UDP Testing: Focused on jitter, packet loss, and connectionless throughput metrics.

     • TCP Testing: Focused on window sizing and reliable delivery throughput.

   • Target Metric: Data transfer rates measured in Megabytes per second ($MB/s$).

   • Installation Considerations:

     • Access Rights: Verification is needed to see if Sai has local administrator privileges for the installation.

     • Service Desk Escalation: If policy restrictions prevent local installation, a formal request must be logged with the IT Service Desk for software deployment.

3. Global Testing and Migration Schedule (February)

   • Budapest Site: Testing sequence initiated today.

   • Chandler and Ashburn Hubs: Commencement scheduled for February $7$ starting at $8$ AM.

   • Risk Mitigation: There is an active debate regarding postponing these windows due to high-risk migration concerns; final approval from the Director is required before proceeding.

   • Provo and Newberry (VPN Cluster): Connectivity testing and tunnel validation scheduled for February $10$ at $9$ PM.

   • Milan, Italy: Configuration confirmation received for a February $13$ window.

   • Germany Core Sites: Parallel testing and validation windows confirmed for February $12$ and $13$.

4. Site Status and Incident Management

   • Lithia Springs Modernization:

     • Status: The site experienced significant instability during the initial rollout but is now classified as "Stable" following a successful firmware/hardware upgrade.

     • Alerting: No active Critical ($P1$) or Major ($P2$) incidents are currently tagged to this location.

   • Legacy Hardware Lifecycle:

     • Cleanup Operations: Ongoing effort led by Brad and Marissa to inventory, decommission, and return legacy equipment to the warehouse.

   • GCP (Google Cloud Platform) Virtual Ions:

     • Architecture: Specialized focus on $IPv6$ design testing within virtualized environments to ensure compatibility with future-state network standards.

5. Incident Classification and Response Framework

   • Priority Redefinition: Proposal to recalibrate the Incident Priority Matrix ($IPM$) to better reflect business impact.

   • P1 (Critical): Reserved for data center outages or total site isolation where the $\text{User Impact} > 80\%$.

   • P2 (Major): Applied when a site experiences degraded performance or loss of redundancy (e.g., secondary branch line down) but remains operational.

   • BGP (Border Gateway Protocol) Triage: A critical point of discussion is whether a $BGP$ flap or peering failure should automatically trigger a $P1$ status given its potential to isolate a site from the global routing table.

6. SLA Compliance and Communication Strategy

   • Documentation Goal: Create a formalized technical flowchart to dictate automated response actions based on the specific alert type (e.g., Latency vs. Down State).

   • Service Level Agreements (SLAs): Improved monitoring is required to track the Mean Time to Repair ($MTTR$) and ensure that vendor (Palo Alto) response times align with contractual obligations.

   • Future Alignment: Continued iterative meetings are necessary to refine the alerting thresholds and ensure the Network Operations Center ($NOC$) can act decisively on high-priority notifications.