Today had a strange issue with a Zondirector 3k and thought I would share in case anyone else has the issue. For some reason all of a sudden the web interface started showing this error:

Request Error

Error logged at the server

Enable EjsErrors to see errors in the browser.

SSH, ping, and AP functionality all still worked great. First we tried to reboot the ZD, but this has no effect. Next we tried to redo the cert – and this was the fix.

First SSH into the ZD with proper user/pass

Ruckus> enable
Ruckus# config
Ruckus (config)# certificate
Ruckus (config-certificate)# restore

This will then reboot the ZD, and APs will connect back – I had a loss in AP connections (clients dropped), and then they all reconnected back.

One great way to troubleshoot wireless issues that clients might be having is to perform a packet capture or use some kind of wireless analysis tool. In this case I have an Ekahau sidekick with Capture. Before this tool I used Ubuntu, Wireshark, and a compatible wireless adapter to capture packets. The MCS or Modulation and Coding Scheme value indicates a lot about the client. For example, the MCS rate is as really a reference number given to the combination of  – amount of Spatial streams, Modulation type and coding scheme. Analyzing the MCS helps when we need to troubleshoot the common problem of “its slow” while everything looks good.

I look at the MCS rate when troubleshooting client problems for a lot of reasons. Since the MCS rate is a reference number describing how much data can be sent over the wireless medium, it gives us an idea of the wireless environment. If the spectrum has lots of contention/interference or the client has low RSSI then the MCS will reflect this. If clients have lots of retrys, or a low RSSI then it will lower the modulation scheme, and therefore lower the MCS rate.

Lets take a look at a 802.11ac packet in Wireshark and break down what each section means. The MCS is located in the Data frame under the 802.11 Radio information section.

This packet is captured from a Data frame, you will notice the MCS rates are only captured on data packets – Lots of reasons for this. Most management/control frames are sent with the lowest data rate possible to allow them to be understood by clients far away, or if the network is very congested.

In this example we are looking at an 802.11ac frame, and we see right away the MCS index is 7  – from looking at the MCS chart we see that means modulation:64 QAM, which a number 5/6 . The 5/6 is the actual FEC or Forward error correction coding rate, which is given in fraction form. 5/6 equals to 0.83333.. which means that 83% of the data stream can be used to send real data. 5/6 is the highest coding rate.

We can see that the host has 2 Spatial streams, is using a Short guard interval , 40 MHz channel and the data rate is 300 – the data rate is  the highest capable with these settings. Notice the RSSI is -44 and we are just using 64 QAM, you really have to be close and have a clear spectrum to achieve 256 QAM.

Below  is the table of MCS rates for 802.11AC/N – Compliments of WLANPROS.com

Fortinac is built on top of CentOS and is a great product. Recently I needed to have default or isolated vlan support PXE booting as well as isolation. This way if a computer is being imaged we don’t have to worry about hard coding ports with vlans, etc. This is important because the NAC cannot look at the client prior to the OS install.

These settings were added to the dhcpd.conf – they would work for any implementation running dhcpd not just Fortinac.

Below is the conf that works.

# Sample /etc/dhcpd.conf

authoritative;
log-facility local6;
ddns-update-style none;
allow bootp;
allow booting;
class “authenticated_clients”
{
match pick-first-value (option dhcp-client-identifier, hardware);
}
# Isolation Scope ISOL_Isolation_blackhole
subnet 172.16.172.0 netmask 255.255.252.0 {
range 172.16.172.10 172.16.175.200;
default-lease-time 28800;
max-lease-time 86400;
option domain-name “blackhole.local”;
option domain-name-servers 172.16.172.254;
option broadcast-address 172.16.175.255;
option routers 172.16.172.1;
next-server PXE-SERVER-IP;
filename “SMSBoot\\x64\\file1.efi”; — NOTICE the two slashes that represent the file path – I was missing this, and of course it could not find the file.

The options for next-server and file-name were the options needed to push PXE settings over.

Restart the service after saving the configuration to dhcpd.conf

This post is password protected. You must visit the website and enter the password to continue reading.

Cisco Flex links give the ability to have a layer2 redundant connection, or pair of connections configured as an Etherchannel for a primary link. This is an active/passive setup where if the primary connection’s link status goes down, the Flex link will become active, and if the primary comes back it will go into a standby mode and not take back primary functionality unless told to do so with preemption commands. STP is disabled automatically on Flex Links so no need to bother with Portfast.

Flex links started in code a long while back, and not sure how I missed them. I have needed this functionality before if I was connecting to a backup Firewall, or some device that Spanning-tree would have issues with. This options gives a great way to have a backup link configured if you just need it to become active if something happens to the primary link. In the below scenario I have a Fortigate firewall and Cisco 3560 switch. Port FA 0/1 is my primary and goes to Port 1 of the FGT, and Port 12 is the backup port for this link.

Layout

Switch config:

interface FastEthernet0/1
description “Connected to Fortigate”
switchport trunk encapsulation dot1q
switchport mode trunk
switchport backup interface Fa0/12
spanning-tree portfast

interface FastEthernet0/12
description “Connected to Fortigate – Backup”
switchport trunk encapsulation dot1q
switchport mode trunk

Below shows that status of the Flex link

Notice that the primary state is active and up. Now, I will cause a physical port state change by unplugging the interface and see how many pings/time it takes to failover.

After unplugging the primary connection, the link light of port 12 instantly came on and went green, I didn’t even lose a ping to my switch. All mac/uplinks moved over the backup port but no loss. Below shows the status after. Notice that the backup state is up, not the primary. The primary port after plugging it back up is amber like a blocked port, and both interfaces port status show up, even though the primary port at this point is not forwarding traffic.

I work with a lot of sites that are multi-homed or have a backup connection that, even though its backup still needs to be used at the same time for load balancing.  Recently I was able to have a site that had a very slow MPLS link that was being moved to only backup – and a new SD-WAN link has been installed as primary.  The MPLS link will go away after were sure the SD-WAN option works well.

The only time we want this link to become active is if SD-WAN fails. My first thought is no problem, just  use a routing protocol and a static route – the routing protocol will be our preferred method, and we can just raise the admin distance of the default route to make it a backup. One problem in this scenario, the MPLS uses dynamic routing (EIGRP), and many other locations still use the MPLS for primary, or secondary connections, so we cannot even have this link up (admin state) or the MPLS will advertise the network.

So, to recap what I need is a preferred route to our SD-WAN, and if SD-WAN fails, bring up the backup connection admin state and move traffic to it. Then of course, auto fix everything if SD-WAN comes back online. No problem! Cisco’s Event Manager to the rescue. Cisco’s Embedded Event Manager (EEM) (Copied From Cisco) –is a distributed and customized approach to event detection and recovery offered directly in a Cisco IOS device. EEM offers the ability to monitor events and take informational, corrective, or any desired EEM action when the monitored events occur or when a threshold is reached. An EEM policy is an entity that defines an event and the actions to be taken when that event occurs. When creating EEM scripts, you have two options TCL or CLI – in this case I am using just CLI.

Using EEM on the 3850 core I was able to detect if the default route learned VIA OSPF was removed from the routing table – if that happened, then run the command “No shut” on my MPLS uplink and EIGRP would take over. If the default route was learned at some point through OSPF when things were corrected – then run the command “Shut” on my MPLS uplink. This worked extremely well in combination with Link detection on the Fortigate, and OSPF default route distribution.

The Fortigate is my SD-WAN device, and my default gateway for the network.  I am using link detection to test HTTP access to google. If my WAN interface cannot get a response from google.com in my set time (5 attempts, with 5 seconds between each attempt) then it will remove the default route from my routing table. When this happens the route will be removed from redistribution in OSPF, and removed from the Cisco core. EEM sees this event, and does my list of commands. Below shows the layout, and code to get this going. Interface 0/24 is my MPLS uplink.

First I made sure that the MPLS interface was  shutdown and OSPF was up, and I was receiving the redistributed default route from the Fortigate.

config t

event manager applet MPLS-UP
event routing network 0.0.0.0/0 type remove protocol OSPF
action 1 cli command “enable”
action 2 cli command “config t”
action 3 cli command “int gig 1/0/24”
action 4 cli command “no shut”
action 5 cli command “exit”

event manager applet MPLS-DOWN
event routing network 0.0.0.0/0 type add protocol OSPF
action 1 cli command “enable”
action 2 cli command “config t”
action 3 cli command “int gig 1/0/24”
action 4 cli command “shut”
action 5 cli command “exit”

You can check status and history of events by using the show event manager commands.

During a failure of my ISP, everything worked great. The default route was removed from OSPF, which caused an event that EEM matched – then it enabled the MPLS interface, and all routes/default was learned VIA EIGRP and the MPLS. When internet was restored, the MPLS interface was shutdown, and all traffic started flowing over SD-WAN.

Recently I was asked to look at a local college that is experiencing issues in the evening with wireless users. In this example the complaints from students who live in the dorm are obscure and lacking details. They range from “Wifi goes out” to “Video’s buffer”, and “Its just so slooooow”. So, with this little information known we decided to go over and check out the spectrum to see what was going on.

First I talked with the admins to find out about the building, construction, Bandwidth/networking and how students use the dorm wireless. From our conversation I found that the dorm has dedicated internet of around 400 MBits and monitoring is show it busy but not maxed out. Students are working throughout the day in their dorms, and complaints pretty much only come in at night. Construction is new, but lots of brick – signal should not propagate to far (this is important for later).  The building which is 5 floors, stands by itself with little sources of wifi interference outside of internal APs.

We then went on site and did a passive survey with spectrum analysis to check coverage, interference,  and if the spectrum was busy.  Wifi coverage with was great, lots of APs to cover needed space, maybe too many APs (we will get to that later). When conducting the spectrum analysis —- WHOA! its not busy, but look at the signals that are seen. Below shows an screen shot of the spectrum using Ekahau.

My first thought was that’s a lot of APs! I looked at our layout and we only had about 6 APs in this area, but I see many more than that. So the idea that the building materials (brick) would attenuate the enough that we would not see APs on other floors was incorrect. This gives a lot of credit to on site surveys before installing APs – building materials might be different and react different then we think sometimes.   I was seeing APs 2 floors above with usable signal. Another thing to notice that there is plenty of open spectrum in the 5 gig range, why are all the channels being used close to each other and at the beginning of the spectrum? At this client Ruckus wireless is being used and after checking documentation and asking vendor reps, seems like it might be a firmware thing. Also notice that we have a lot of APs on the same channel that can hear each other, this would cause contention on the channel.

This shows the importance of always doing a site survey and checking spectrum to see what AP coverage is.

To help resolve the issues we created a channel plan to statically set channels and spread them throughout the spectrum, also lowering channel width to 20 MHZ since we have so much AP coverage between floors. By doing these two things we were able to decrease channel contention and give our users a better wireless experience.

One additional setting we modified was to block broadcast/multicast from clients. When doing a packet capture on the spectrum we also noticed that around 30 percent of the traffic was just MDNS. We created ACLs to block this traffic.

When deploying wireless networks one of the best practices is to limit the SSIDs used within the WLAN. The reason we want to limit the amount of SSIDs, is the SSID announcement is sent through a Beacon frame. This frame is a broadcast to all stations. This happens every 100 MS or 102.4 MS according to how busy the medium is. In all documentation and great blogs I have not found a lot of visual examples of what actually happens, So I thought this might be a cool blog entry to write.

On top of the constant beacon broadcasting of the APs, when a client wants to join, it will send a probe request to the broadcast address of ff:ff:ff:ff:ff:ff and all APs will respond with a probe response frame directly to the station informing the client of the SSID and its supported capabilities. Beacons and probe request/response frames are the reason we need to limit the amount of SSIDs. For instance (and I will show this below) if you had 1 AP with 5 SSIDs, with 1 client trying to connect, the AP would be sending somewhere around 50 Beacons in 1 seconds, and 5 probe responses. This is might not seem like a lot of air time used at first, but think if you had 5 clients trying to connect at the same time, or maybe 2 or more APs on the same channel, with 5 clients trying to connect at the same time. Then the airtime usage goes up huge. Since wireless is half-duplex this takes increases the time it takes for clients to do what they need to do on the spectrum.

To show the results, I am using the Ekahua sidekick with spectrum analysis. I will start with 1 AP broadcasting 1 ssid, and then add another AP on the same channel. From there I will increase the SSIDs and show the airtime usage.  I hope to show why it is important to keep SSID overhead to a minimum. The gear used will be a Ruckus Zondirector, Ruckus 7982, and R710 APs. I am lucky enough to have an area with very little neighboring wifi networks, so I am testing with 1 wireless client in range of the AP to limit, and show the probe request/response traffic. I am broadcasting the test SSIDs on channel 11 and channel 157.

First lets look at just 1 AP with one SSID broadcasting.

As you can see from above, very little spectrum usage. Again, no clients are connected to an SSID, and only one device would be doing probe requests.

I did a packet capture next, check out the analysis data and the size of the capture. I only let the capture run for 1 minute (60 seconds). The size of the file was roughly 165 Kb. The red in the graph below represents beacon frames, and the dots/green are probe requests vs probe responses.

I then added the other AP on the exact same channels, still only broadcasting 1 SSID. Spectrum usage definitely increased but only by very little. Below are the spectrum density graphs and Wireshark IO graph.

Now, lets increase things a bit – I will start broadcasting 4 SSIDs from 1 AP.

Spectrum usage from 1 AP broadcasting 4 SSIDs:

Below is 2 APs broadcasting 4 SSIDs – all on the same channels.

When comparing the wireshark IO graph we can really see how many Beacon frames are sent and compare those between 1 and 2 APs. Another cool thing to note is the size of the wireless PCAPs – each running for 1 minute, the 1 AP broadcasting 4 SSIDs size was 662 KB, and 2 APs broadcasting 4 SSIDs each was just about double at 1200 KB. In the graphs below, red indicates the number of Beacon frames.

2 APs – 4 SSIDs:

After all the testing, we have a big increase from just adding 1 SSID and an even bigger increase from having another AP broadcasting on the same channels. This isn’t ground breaking but cool to see in action. I have seen many many clients have at least 6 SSIDs broadcasting, and this give some good illustration on why we need to reduce the amount of SSIDs in use. There are a few tools out to calculate the SSID overhead – tools such as http://www.revolutionwifi.net.

Check out the below picture comparing 1 SSID to 4 SSIDs just from 1 AP for 2.4 and 5 gig.

.

Thought this might be useful for anyone who needs to make a lot of changes quickly to many vlans. In ICX vlans, and vlan interfaces (routing interfaces) are different. In this example I will show how to quickly edit each vlan in a range and modify STP settings.

Sacred-CORE#show vlan brief

System-max vlan Params: Max(4095) Default(1024) Current(1024)
Default vlan Id :999
Total Number of Vlan Configured :29
VLANs Configured :2 32 50 to 51 128 132 136 138 140 142 144 146 150 152 154 156 160 162 164 220 224 240 250 255 324 332 350 500 999

As you can see we have a lot of Vlans, so lets modify the STP settings for all of them. I will just copy the vlans listed above and basically paste them in since I want to modify all vlans.

Sacred-CORE#config t
Sacred-CORE(config)#vlan 2 32 50 to 51 128 132 136 138 140 142 144 146 150 152 154 156 160 162 164 220 224 240 250 255 324 332 350 500
Sacred-CORE(config-mvlan-2*500)#spanning-tree 802-1w

Sacred-CORE(config-mvlan-2*500)#spanning-tree 802-1w priority 4096

Sacred-CORE(config-mvlan-2*500)#

Bam! now we have modifed the setting for all vlans. The syntax of the command is

vlan X X X X – so lets do vlan 2, 128 and 136

vlan 2 128 136   — thats it.

The old Procurve switch line is very long in the tooth, but I run across them all the time. In this case its a 2520 switch that is in very bad need a of a firmware upgrade. I will detail where to go to get the firmware, and how to use TFTP to upload it. In my case, the web interface is having all kinds of java issues, and TFTP is just easier.

First lets get the software – it can be found at :

https://h10145.www1.hpe.com/support/SupportLookUp.aspx

Next, I download the firmware for the J9299A, and it seems the latest is from 2016. I download it, put it on my TFTP server which has full network connectivity to the switch.

SSH to the switch, and run these commands

If you get the error ” SFTP must be disabled before enabling tftp. ” you will need to run the “no ip ssh filetransfer” command first before enabling tftp client.

config t
no ip ssh filetransfer — This allows TFTP, if you enable this, and use SCP or SFTP then no need for TFTP
tftp client
exit
copy tftp flash 10.10.16.5 J_15_09_0028.swi primary

It will then write this to the primary flash. Next I will tell it to boot this firmware.

boot system flash primary  – Now it will reboot with the latest firmware
AH-POE-Top# show ver
Image stamp:
/ws/swbuildm/J_rel_hartford_qaoff/code/build/walle(J_rel_hartford_qaoff)
Aug 23 2016 08:57:14
J.15.09.0028
1791
Boot Image: Primary

Finding what vlans are set on a switch port is a very needed thing for almost any config changes in Procurve software. This entry shows a quick way to check the vlans both tagged/untagged on a procurve. This works for all procurve I believe, but I am testing on a J9773A 2530 switch. This is a simple entry but might help someone out.

To show vlans associated with the ports the command “show vlan ports X” can be used, and to find out more info like tagged/untagged you can add the “detail” to the command to get more info. For example to get info for port 1

Show vlan ports 1

And more info:

show vlan ports 1 detail

This entry details how to create a static route in both the GUI and CLI of the Fortigate firewall. Specifically I am using FortiOS 6.2.4 but its pretty much been the same for years.

Lets start by talking through the things that will be needed to create the static route.

– Subnet – this is what we want to route to, for a default route its 0.0.0.0/0 but if we wanted a more specific route, lets say to 192.168.100.0/24.

– Destination Interface – Next hop interface we want to send traffic out of.

– Gateway address – Directly connected interface neighbor that we want the next hop for 192.168.100.0/24 to be.

– Administrative Distance– is a feature used by routers to select the best path to a destination when multiple paths to the same destination are present. Lowest AD wins and will be placed in the routing table.

Advanced option – Priority – To build on AD definition – What if two routes exist in the routing table to the same destination with the same AD? This is where Priority comes in. Lowest priority wins. By selecting a priority you can have multiple routes to the same destination in the routing table, but one would be preferred over the other. This comes in very hand for Reverse Path forwarding issues.

So after all that’s said, we need to route 192.168.100.0/24 to our LAN interface with a next hop of 192.168.1.2.

First lets create this in the GUI. Navigate to network – static routes – and create a new one.

.

Now we will just insert the needed info. I am leaving the AD at 10 – which is default.

Press OK – and Bam! route created. We can check that the route has been created and is the routing table by going to monitor – routing monitor.

Next lets do the same thing in CLI.

First route creation. When you create the route edit the next available sequence number. In this case its 46.

You can see if your route is in the routing table in CLI by running the command “get router info routing-table all” but in this case I am using the static option, and grepping just what I need to see.

Overtime we forget things, especially Shared secret radius keys. This is pretty common, and I run into it a lot. For example – lets say a you setup NPS (Network Policy Server) and a Wireless controller for 802.1x auth, or a ASA doing radius authentication years ago. Some how or another that key was lost – no worries, you can get that back from the NPS server itself.

In just a few simple steps you can get that key back. So lets start by opening up NPS and then selecting “Radius Clients and Servers” and dropping down “Radius Clients”

In this example I am using a Ruckus Smartzone – lets say I forget the password. I can just right click on the client and select “Save and apply as Template.

Next we can create a new radius client by right clicking on “Radius Clients” and once the client info pops up to fill in, we will select to create it from the template, and select the template we made.

To see the *** Password, uncheck the box “Select and existing template” and then select the “Generate” Radio button – and bam! there is the PSK.

Had a strange issue the other day with a FAC, where it would not send emails to users with their assigned tokens, but would send emails just fine any other time. I wanted to capture all outgoing traffic to see if SMTP messages were really being sent.

Fortiauth has Tcpdump built in, and is very easy to run.

First SSH into the FAC, from there you have some execute options. Below shows the tcpdump options:

exe tcpdump?
tcpdump Examine local network traffic.
tcpdumpfile Same as tcpdump, but write output to a file downloadable via GUI.
exe tcpdump

If you run ‘exe tcpdump’ it will spit all the traffic to the screen, but if you run ‘exe tcpdumpfile’ it will log the output to a .pcap that is downloadable from the GUI. This gives you the option to open it in Wireshark and analyze.

To download the .pcap open your Fortiauth append /debug to the web address for example: https://10.110.2.60/debug. From here you will be prompted with what you want to debug, and at the bottom is the option to open the “CLI Packet Capture” this gives you the option to download the pcap.

Thats it! Thank you Fortinet.

When you setup Fortinac you have to license it, and Fortinet asks you what the MAC and UUID of the device are when registering the license. You can get this information by SSH’ing into the NAC and running the following commands:

 sysinfo -v | grep -i UU   — This will bring back the UUID

and to get the mac – run  ifconfig eth0

Copy those two settings into the registration of the license, and you can then get the license key.

This entry shows how I have been setting up ICX switches with Fortinac.

In this scenario my Fortinac is located at 192.168.226.248, the switch is 192.168.226.53, and my SNMP community is “snmp”. I know very secure. The switch I am working with is a Ruckus 7250 running SPR08092a.bin

These are the settings that I am putting into my switch:

logging host 192.168.226.248
snmp-server host 192.168.226.248 version v2c snmp

snmp-server enable traps mac-notification
snmp-server enable traps link-up
snmp-server enable traps link-down

On the NAC we have to add the switch, and make sure we have a CLI user account, and SNMP creds that work. We can test this within NAC to make sure things are up and going.

After we add the device, we can validate the settings

After the device has been added you should see your interfaces/devices/status all show up.

Certificates for VPN, SSL Offloading (if using Load balancing), or a signed device cert expire, we all know this. Up until last week I had never updated a signed certificate, I had just created a new CSR, and rekeyed the cert. Updating the certificate the Fortigate is using is very easy, but I had problems with the syntax so I am documenting it here.

The Fortinet KB article to do it is located here:

https://kb.fortinet.com/kb/documentLink.do?externalID=FD35074

I had an issue following the doc so I though I would clear the water and see if I could help someone down the road. Lets say I have sslvpn.travelingpacket.com that will expire in 2 days – I log into my CA (godaddy in my case) and renew the cert. They send the new cert to me, but what do I do with it…

Open the cert with a text editor – maybe notepad – and copy the cert. you should see —BEGIN CERTIFICATE. Copy everything. Then log into the fortigate VIA cli – Putty or some kind of SSL client is way better for doing this then the web client. Then lets modify the certificate

config vpn certificate local

edit sslvpn (or your cert name)

set certificate “—–BEGIN CERTIFICATE—– mPjDQDYkYHKcTrGa6aH7e1w1uM7kdaBAjyAgM7xcmuTrsCeLYfd+BwIDAQABo4IDTDCCA0gwPQYJKwYBBAGCNxUHBDAwLgYmKwYBBAGCNxUIorRWhO7dYIKtkziB9KY0
>—–END CERTIFICATE—–“

and Press enter – The issues I had was with the quotes. I tried to first do double quotes, and past the cert in the middle – that does not work. Just simply type in the command set certificate and then a double quote “ and past the cert whole. After its pastes do the ending quote “ and press enter. Thats it for modifying the cert – but to enact it we have to remove it from whatever we are using it for, and then add it back. That refreshes the cert. So if your using it for SSL-VPN , go to VPN – SSL-VPN settings – and set the server cert to a different one, press apply, and change it back.

I am more impressed with Fortiswitches every time I work with them. The ability to implement light NAC features, INTRAvlan firewall policies and overall management really gives these switches a feature set to checkout when deciding on new switches.

Below are the steps to quickly get the interface stats such as errors/packets, etc. The commands are ran on the Fortigate, which in this case is controlling the Fortiswitch.

Drop into CLI on the FGT and check what switches are connected by running the command

get switch-controller managed-switch

This command will bring back the names of the manged switches. Locate the switch you want to check the port stats on. For example, we will use the name “FS1D24T419001174”

the command to get the stats are:

diag switch-controller switch-info port-stats FS1D24T419001174 port1

The output is in the image below:

using the top level command diag switch-controller switch-info you can also get LLDP, Power, and lots more info of the managed switch.

I had an issue, well more of a specific formatting issue with Fortiauthenticator that I thought I would share. I have a client who is only use SMS with forticlient via fortiauth. The idea is that the user connects and authenticates to the SSL VPN, and then hits Fortiauth for token code that was sent to the client VIA SMS.

When using SMS with tokens, you have to have the users mobile number entered so it can send to them. Hard coding the users mobile number worked great, but for some reason I could not get the remote sync rule to pull in the mobile phone number. Below are the steps I used to fix this.

First in the remote sync rule under “LDAP User Mapping Attributes” modify the mobile data field with “mobile” all lower case.

Then make sure that in Active Directory the mobile number is entered under the users profile. the Auth says it wants the mobile phone number in a very specific format – +[international_number] – this threw me for a while. In the end the number in AD wasn’t the problem it was the mapping attribute. Below is how to inset the number into AD. Notice the number has +[country code]number. Thats it, after putting that in the remote sync rule worked fine.

I have been working a lot with the Dell N-series over last few years, and now the N2248-ON which can run OS10 as well as the default OS6. We upgraded firmware to the latest 6.6.3.10 and all seemed to go well. Somehow it did not and hosed both primary/secondary firmware. The device was boot looping – the only option was to drop into ONIE Recovery and re-install the firmware. Here are the steps I used:

The ONIE recovery area runs a version of Linux. First check out your NIC to make sure it finds it:

ONIE:/ # ifconfig
eth0 Link encap:Ethernet HWaddr 8C:47:BE:97:B5:0F
inet6 addr: fe80::8e47:beff:fe97:b50f/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:67 errors:0 dropped:0 overruns:0 frame:0
TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6904 (6.7 KiB) TX bytes:1198 (1.1 KiB)
Memory:dfe00000-dfe7ffff

Great! Eth0 is found, but of course link status is down. Eth0 is the out of band management interface. We should be able to set an IP address on the interface and install firmware VIA TFTP or USB.

First I will setup and IP that can communicate with my laptop :

ifconfig eth0 192.168.1.100 netmask 255.255.255.0

ONIE:/ # ifconfig
eth0 Link encap:Ethernet HWaddr 8C:47:BE:97:B5:0F
inet addr:192.168.1.100 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::8e47:beff:fe97:b50f/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:108 errors:0 dropped:0 overruns:0 frame:0
TX packets:49 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:10874 (10.6 KiB) TX bytes:7658 (7.4 KiB)
Memory:dfe00000-dfe7ffff

Perfect! My laptop is 192.168.1.99 – and connected directly to the out of band MGMT port.

Next we will TFTP the file up. This file is located in the software archive you download from Dell – Its located in the
“Otherfiles” folder. In this case the file name is onie-installer-x86_64-dellemc_n22xx_6.6.3.10. Next I put this on my TFTP server and we can start the install.

First lets turn off the ONIE-Discovery attempts with the onie-stop command.

Now lets upgrade – Run:

onie-nos-install tftp://192.168.1.99/onie-installer-x86_64-dellemc_n22x
x_6.6.3.10

The onie-nos-install will install the OS back to the device. The firmware took a few minutes to install, with about 4 reboots I think – it was all automatic.

Now lets see if the switch was updated with the “show version” command.

All is good!