Juniper NFX250 — disassemble and initial setup (Part 1)

By | 23.01.2021

Recently I got Juniper NFX250 for tests, will disassemble it and figure out what we can run on this unit.
Taking off the hood to see what’s inside:

I will not take off the radiators, will try to learn from console which chip is hiding there.

So, we can see a mini server with an integrated packet processor from Broadcom.

On this model – Juniper NFX250 ATT LS1 :
CPU – 4 core Intel(R) Pentium(R) CPU D1517 @ 1.60GHz
RAM – 16Gb DDR4
SSD – 100Gb
Apparently, adding memory or replacing disk will not be difficult. The most sore point is the CPU. It is hardwired to the board and cannot be replaced.
Here you can read the datasheet for the NFX250.

How does the Juniper NFX250 works?

There is Ubuntu 14.0.4 based hypervior under the hood.
Virtual Machines are working under KVM virtualization.

NFX250 is based on the following key components:

JDM (Juniper Device Manager) – Linux container for resource and virtual machines management, basically this is Juniper’s JunOS like virsh adaptation.
VM is called “jdm”.



JCP (Juniper Control Plane ) – Junos VM used for Data Plane configuration – ports, vlans etc.
VM is called “vjunos0”.


VNF ( Virtualized Network Function) – Virtual Machine or service. You can run anything as long as there are enough resources.
For example – vMX, vSRX, freeware FRR, Vyos etc.

Juniper NFX250 basic setup

Before starting the tests, I will install clean JUNOS 18.4R3.3

Let’s start with the first launch.
The first step is management sеtup. I will use the front panel MGMT port. This is the Out-of-Band management port and it is connected to the dedicated management bridge. The JDM jmgmt0 port is connected to the same bridge.


The first thing we will see in the console is JDM. Checking the list of running virtual machines:

root@jdm:~# cli 
{master:0}
root@jdm> show virtual-network-functions 
ID       Name                                              State      Liveliness
--------------------------------------------------------------------------------
2        vjunos0                                           Running    alive
11525    jdm                                               Running    alive

{master:0}
root@jdm> 

Let’s set up management on JDM by enabling DHCP and setting up a password for root. By default, jmgmt0 is configured with the address 12.213.64.65/26:

root@jdm# show interface jmgmt0 
vlan-id 0 {
    family {
        inet {
            address 12.213.64.65/26;
        }
    }
}

root@jdm# delete interface jmgmt0 

{master:0}[edit]
root@jdm# set interface jmgmt0 vlan-id 0 family inet dhcp 

{master:0}[edit]
root@jdm# set system root-authentication plain-text-password    
New password:xxxxx
Retype new password:xxxxx

{master:0}[edit]
root@jdm#  set system services ssh root-login allow

{master:0}[edit]
root@jdm# commit 
commit complete

{master:0}[edit]
root@jdm# quit 
Exiting configuration mode

{master:0}
root@jdm>  show interface jmgmt0 
JDM interface statistics
------------------------
Interface jmgmt0:
--------------------------------------------------------------

Link encap:Ethernet  HWaddr b0:33:a6:33:f3:cb


inet addr:10.0.17.193  Bcast:10.0.17.255  Mask:255.255.255.0

Ok, the management interface of JDP has got dhcp address and now we can log in via SSH. Let’s move on to setting up JCP management. By default, em0 is configured with the address 12.213.64.66/26:

root@jdm> ssh jdm-sysuser@vjunos0
Last login: Tue Jan 12 10:37:41 2021 from 192.0.2.254
--- JUNOS 18.4R3.3 Kernel 64-bit FLEX JNPR-11.0-20191211.fa5e90e_buil
{master:0}
jdm-sysuser> configure 
Entering configuration mode

{master:0}[edit]
jdm-sysuser# set system root-authentication plain-text-password 
New password:xxxxx
Retype new password:xxxxx

{master:0}[edit]
jdm-sysuser# set system services ssh root-login allow 

{master:0}[edit]
jdm-sysuser# delete interfaces em0 
jdm-sysuser# set interfaces em0 unit 0 family inet dhcp 

{master:0}[edit]
jdm-sysuser# commit 
configuration check succeeds
commit complete

{master:0}
jdm-sysuser> show dhcp client binding 
IP address Hardware address   Expires State   Interface
10.0.17.174 b0:33:a6:33:f3:b8  86391  BOUND   em0.0                  

The JCP management also received an address via DHCP.

ASIC used in NFX250

Basic management setup id done, now we can login to JCP over SSH and see which chip is hiding under the radiator.

{master:0}
root> start shell 
root@:RE:0% vty fpc0


Switching platform (1599 Mhz Pentium processor, 255MB memory, 0KB flash)

FPC0( vty)# set dcbcm diag
Boot flags: Probe NOT performed
Boot flags: Cold boot
Boot flags: initialization scripts NOT loaded
BCM.0>
BCM.0> show unit

Unit 0 chip BCM56063_A0 (current)
BCM.0> 
BCM.0> show params
#######OMMITED############
driver BCM53400_A0 (greyhound)
unit 0:
        pci                     device b063 rev 01
        cmc used by pci         CMC0
        driver                  type 132 (BCM53400_A0) group 90 (BCM53400)
        chip                    xgs xgs3_switch xgs_switch 
#######OMMITED############

####DO NOT USE "Exit command here", hit ctrl+c instead#####
BCM.0> exit on signal 2
root@:RE:0% 

Based on the PFE console output, a Broadcom BCM56063 chip is hidden there. It was not possible to find an open datasheet for this chip.

Is it possible to run BGP router on JCP ?

Since JCP is powered by Junos, I was wondering if it possible to use it as a BGP router with Full View peers.
I have tried to fill in BGP Full table. Platform FIB was exhausted and following error thrown:

Jan 13 14:21:05 fpc0 brcm_rt_ip_uc_lpm_install:1372(LPM route add failed) Reason : Table full unit 0
Jan 13 14:21:05 fpc0 brcm_rt_ip_uc_entry_install:1230brcm_rt_ip_uc_entry_install Error: lpm ip route install failed vrf 0 ip 1.20.128/18 nh-swidx 1734 nh-hwidx 100007

JCP is limited to 1000 lpm routes.
Changing the profile to “l3-profile” does not affect the situation in any way:

root> show chassis forwarding-options 
re0:
--------------------------------------------------------------------------
UFT Configuration:
l2-profile-three.(default)
num-65-127-prefix = 1K

{master:0}
root> show chassis forwarding-options    
re0:
--------------------------------------------------------------------------
UFT Configuration:
l3-profile.
num-65-127-prefix = 1K

{master:0}
root> show pfe route summary hw    

Slot 0

Unit: 0
Profile active: l3-profile
Type            Max       Used      Free      % free
----------------------------------------------------
IPv4 Host       4096      7         4085      99.73
IPv4 LPM        1024      3         1019      99.51
IPv4 Mcast      2048      0         2043      99.76

IPv6 Host       2048      2         2043      99.76
IPv6 LPM(  64)  512       1         510       99.61
IPv6 LPM( >64)  0         0         0         0.00
IPv6 Mcast      1024      0         1022      99.80

Ok, JCP cannot be used as regular MX router )
We can dont install routes to FIB and use this workaround in following cases:
– Use JCP as a route-server
– Forward Full View table(in case we have just one upstream)
Let’s try:

[edit interfaces xe-0/0/13 unit 0]
+     family inet {
+         address 172.16.23.2/30;
+     }
[edit protocols bgp]
+    group test {
+        type external;
+        export reject;
+        local-as 65234;
+        neighbor 172.16.23.1 {
+            family inet {
+                unicast {
+                    no-install;
+                }
+            }
+        peer-as 65000;
+        }
+    }

With this configuration, the session is established, routes are installed to RIB only. Filling the complete table (830k prefixes) takes 4 minutes.

root> show bgp summary      
Threading mode: BGP I/O
Groups: 1 Peers: 1 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
inet.0               
                  830103     830102          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
172.16.23.1            65000     139620         15       0       1        5:45 Establ
  inet.0: 830102/830103/830103/0

Only 7% of free memory left:

re0:
--------------------------------------------------------------------------
System memory usage distribution:
       Total memory:  476164 Kbytes (100%)
    Reserved memory:   13388 Kbytes (  2%)
       Wired memory:  132664 Kbytes ( 27%)
      Active memory:  267656 Kbytes ( 56%)
    Inactive memory:   27840 Kbytes (  5%)
       Cache memory:       0 Kbytes (  0%)
        Free memory:   34612 Kbytes (  7%)

Memory settings of the JCP virtual machine can be edited from the hypervisor:

root@jdm:~# ssh hypervisor
Last login: Wed Jan 13 09:27:25 2021 from 192.0.2.254

root@local-node:/# virsh list        
 Id    Name                           State
----------------------------------------------------
 2     vjunos0                        running

root@local-node:/# virsh edit vjunos0

Moving to next step – practice tests.

NFX250 Performance tests

The NFX250-S1 has the following specifications:

Managed Secure Router*** 2 Gbps
Managed Security*** 2 Gbps
IPsec*** 500 Mbps
*** Maximum throughput mode

We have NFX250 ATT LS1, it has 2 fewer cores.

Let’s try to test the chip by passing traffic through the JCP.

Performing reference test by connecting the servers directly:

# iperf3 -c 10.10.0.2 -p 5201 
Connecting to host 10.10.0.3, port 5201
[  4] local 10.10.0.2 port 42376 connected to 10.10.0.3 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  1.10 GBytes  9.42 Gbits/sec    0    482 KBytes       
#######OMMITED############
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  11.0 GBytes  9.41 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  11.0 GBytes  9.41 Gbits/sec                  receiver

Lab scheme:

Now I run the test through JCP:

[s1]# iperf3 -c 10.223.0.2 -p 5201
Connecting to host 10.223.0.2, port 5201
[  4] local 10.222.0.2 port 58338 connected to 10.223.0.2 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   365 MBytes  3.06 Gbits/sec  281   73.5 KBytes       
#######OMMITED############
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  3.23 GBytes  2.78 Gbits/sec  2742             sender
[  4]   0.00-10.00  sec  3.23 GBytes  2.77 Gbits/sec                  receiver

The throughput is 2.78Gbps. Not bad for a platform like this.

In the next part, we will deploy virtual machines with FRR and Vyos as VNF and repeat the throughput tests.

Leave a Reply

Your email address will not be published.