Kentik - Network Observability
More episodes
Network AF  |  Season 1 - Episode 3  |  October 12, 2021

Welcoming newcomers to the networking industry with Janine Malcolm

Play now


Today's conversation is with Janine Malcolm, Director of Network Engineering at Salesforce. Janine takes us through her journey to get to where she is today and how she became interested in networking itself. Not only do they get into the nuts and bolts of networking, but also what you can do to get into the industry and why having a college degree isn't always necessary. Janine shares how we can make this space more welcoming to newcomers with advice on how you can start learning more and get your career going.


Welcome to this episode of Network AF, and I'm proud to welcome my friend and fellow networker, Jeanine Malcolm. Janine, if you could, introduce yourself and tell us a little bit about, what you're up to in networking.

Hi, Janine Malcolm. Salesforce director of network engineering.

We run big networks.

So, What am I doing right now in network engineering?

Running, couple of groups at the moment.

We have, the backbone team that was kinda mushed together into a overall network engineering team. And so we also have another team that helps this friend some of the, M and A networks as well, underneath me is now.

So, yeah, it's been a big changed for me, because the first couple of years, it was just the backbone.

And now it seems to be everything else. So it's pretty cool.

Well, I mean, it's interesting because in service providers, they just call the network of the network because that's Yeah. Revenue. An enterprise, it's like, there's prod production networking, which one's the revenue. And of course, without Salesforce, people can't find each other in business nowadays.

And they can't look up who said what to who and all the other systems failed because he was system of record. And then there's corp or, you know, IT, which is different kinds of networking, and, figure mostly on the on the prod side there. We're on the prod side. Yeah. And the and there is a business pathology group, and they have an entire wonderful team that helps on that side.

But no, we're just on the production side. Yeah. So different customers, same idea, different scale, maybe, of of the prod side. I know. They have a lot of offices.


Well, it's more location, maybe less volume, definitely. Applications.

Cool. So so what made you interested in networking? Like, where where was where Where were you when you decided networking was interesting or what sort of got you into it? I was in college, and I didn't do so well.

It wasn't that I didn't do so well. I I felt like it was not very good education for my money because I was paying for it myself. And so I decided, working full time and going to school full time was a little bit of a challenge. Mhmm.

So I decided, okay. Well, what am I gonna do? And I was actually working on a farm that I had worked on as a teenager when a friend told me that they were interviewing at Digex. And I said, well, Hey, I've always loved computers. My my my grandfather actually got me into computers really young, like four or five. Oh, wow. Yeah.

And so, like, I was like, okay. Well, let's try this out.

I'm sure I can do this. And I went in as, customer service person in Digex, and dial up. So, and then I moved to, like, leased lines to, like, t ones, you know, that kind of stuff. T threes, that kinda thing. And then I just kinda gradually I was over on the web server side for a while. That was eye opening for me.

It's new for everyone at the time, you know. Yeah.

Especially with what web servers were mainly doing at that time. Yes. We'll see. We'll move on.

And then, I went and set. And yeah. Yeah. A couple two years in, I was over on the network engineering side.

And, I really, you know, I really decided to, wow, this is really cool. I love this. This is based like, a big picture, and I get to move data around. And I'm, like, you know, learning how to at least code and parole, hey, look.

Sorry. I still do. I still do. I I still do. I still I I admit it.

I haven't even started reforming yet.

But that's how I got into it. Yeah. No. That's cool. I mean, I, I remember my first day of computer science And I bet I don't have a degree.

I I left before. And, I met. The person did image processing, and I said, oh, that's really interesting. And the guy that did networking, I'm like, oh, Okay.

And then, you know, I found I was interested in, you know, parallel distributed, found the internet, you know, back at a good time. And I also like you, I'd been fortunate. And then when I was eight, you know, my uncle gave me a book on basic, and my father was doing, you know, did stuff. And My uncle briefly let me on the ARpanet, and I quickly found systems he didn't think I should be in.

So that was my, like, early early intro was like, no, no, change password, stay away. So we have fond memories of DigX. Not only because Doug's an awesome guy, and Lisa and, you know, Ed and and, and Dave McGuire and the whole crew, because I was, at Disclave, which was the science fiction convention. Yes.

Yep. My wife and I Gail and I met at World Con in Chicago a few years before that. And I saw this flyer, you know, some three, two sixty, you know, unix, shell access, I was like, I could do that. I have a son for in my basement.

And so we did that. I'm actually partnering with Digex, a little bit. And of course, Digex you know, it's pretty early in growing backbone, which then Exactly.

Well, no. It's it's, it was pretty cool.

So, and and it net access of my ISP, people started often sort of in the knock. Be when we had a network before that, it was, yeah, support. That's always interesting because at the time, it was, like, people who had bright shiny eyes would either be like, oh, systems network or there's some people that really are passionate about support and god love them because I've done my share support and I love talking to customers.

But, you know, it's like, you know, they can't use your door because their disc is full can be frustrating sometimes as you remember.

But, yeah, I mean, that was that was sort of a early group. And I guess we'll later talk about sort of how do we help people get in? Because was a pretty cool time to come in and anyone that had the bright shiny eyes was like, you know, it was helpful. So I guess, you know, what was gonna get opportunity.

Like, what was really helpful, you know, in digitax, you know, sort of earlier in your your career?

I guess two part question was really helpful, and you know, what was, like, frustrating that we could do better at, you know, and in terms of welcoming people.

Sure. So one thing that was fantastic is they and I've I've tried to do this as much as possible with everybody in all of my teams now.

Is that it was a big mentoring experience.

Like, I literally sat with someone, and we just I was, like, right over their shoulder, and they were right over my shoulder. We were, like, working on problems and, you know, doing stuff together. Hey, let's go do this router over here. Hey, let let's go troubleshoot this problem. It was honestly fantastic that way. And it is you can still do that even in the times of the pan pandemic, but I think it was amazingly helpful. One other thing that I think was wonderful is that, you know, Digix had a lot of fantastic women.

And I actually not Doug's wife is an amazing woman. Exactly. Exactly. Right? And then there is Meghan, who is my my mentor.

So, like, I I remember finally, and then Tory, Morton focused now. So, like, you know, it was really great to have so many, you know, people to help out. It was a lot of helping each other. It's just, I think, great.

I I I still feel like that is the biggest way to go. I think you learn I think what is it, like, seventy percent you know, you're sitting there, you know, experience and mentoring and everything. And we'd still do that on all my teams. We cross train.

I think that's a fantastic thing. I actually signed up to be a mentor because I I really wanna help. And even though I know I'm not gonna have the same technical skills person who I'm probably mentoring because we do, you know, it's like Right. Okay.

Like, like, software mostly software developers. Right? And I'm and I'm not. But, still I don't learn Python even if we think that White Space shouldn't be syntax.

So it's I thought the other day. That's so funny.

Well, tickle fortran, I mean, you know, it's it's not a not an illustrious line of white spaces syntax, but, you know It's true. It's true.

But, yeah, no. I think that I think that that same thing is a huge even now. Mhmm.

Definitely. Well, I guess that's an interesting question because, as the world gets more complex, You know, it's like, a friend of ours from IBM Cloud says without the network, there is no cloud or the Zen version, which is network is the water of the clouds, the connective tissue, clouds are made of. But, you know, we have these amazing computers, these amazing networks and these amazing SaaS services like Salesforce.

And, you know, it can be hard for someone in in college or even not going to college to sort of get at what all the underneath is, and then we have this tribal knowledge.

So I guess that's an interesting question. Are you you're still trying to at Salesforce, keep those mentorship patterns, maybe not pair programming or or routing or whatever. But, you know, even even with COVID and maybe the after, I think it's important to do that. I think, you know, even though, yes, we have all these wonderful classes and everything, and it's way better than it was, like, when I was in college, there were not it wasn't network engineering.

There was, you know, computer science, and that was pretty much it. Like, right? And so studying protocol fields of things, not dynamics of distributed networks. Right?

Exactly. So it's a different thing. And and I think that there are some really good programs these days. But I still think that you know, the good old fashioned mentoring is fantastic and and it it applies to a lot of different kinds of Right.


Right? So let me ask you a question about that.

So, I will say this to someone who has destroyed the internet one and a half times because I was associated with Vinnie and AS seven thousand seven. I was the fat man and the fat man little bit internet exchange, but it wasn't my thing. This is Jay router. Have you ever blown up the internet, Janine?

Oh, heck. Yeah. Okay. So Who hasn't, like, taken BGP off of a router? Or inverted their your access list or Yes.

Something. Yeah.

Fifty five two fifty five zero zero zero zero We said we weren't gonna make vendor jokes. So, you know. Sorry. Yeah.

So, I was at a I was actually a social last night where he's like, oh, you were you an ISP? Did you know the internet sometimes gets blown up? I'm like, yes, I'm aware. But you'd think that I mean, a lot of this speaking of, like, Old Young getting into it, it it feel feels like we all had permission to sort of explode the internet. And, like, now if someone at Kentech said, hey, I might try this thing, and it might explode the internet. It might take our service down.

I'd be like, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no. I don't you do.

Or, or, I wish I had had better labs then. Honestly, I I do honestly think that labs these days are much better.

You know, I did I was actually in the lab at DigentX for a while in helping build the lab and all that stuff. But it wasn't like it is today. I'm able to recreate a scenario a bit better. And I I do think on that front, if you're talking about big changes and that kind of thing, automation, getting people off keyboards, Yep.

That's all good. I mean, you're gonna have human errors. Like, you know, that people get tired. They get exhausted.

You're cramming more things in the smaller windows and that kind of stuff to make sure things, you know, stay up. But in order to keep things up, you gotta actually change and fix them and stuff. So you're gonna have human errors. Yeah.

And the goal is to have computers automate the good and not replicate the bad, you know. Exactly.

But, I mean, people may not know this, but when Jean and I started doing networking, there were no comments on static routes. And most networks were bubble gum and static routes, and no one knew why it was there, so don't touch it because things might break. So definitely I mean, the ability to commit rollback and get Oh, rollback.

My best friend.

Yes. I mean, the old way was reboot in fifteen minutes and then make the change, and then I hope that you've remembered to cancel the reboot earlier. You whatever you call that. So Yep. No. That's cool. Were there any frustrations, any things that you think?

You know, we could be doing better you know, at the time, you know, or or or now to make things welcoming for people getting into it.

I think at the time, because it it was really word-of-mouth.

And I wonder how much word-of-mouth there is now. I really feel like there is.

It's much it's still that way. Still word-of-mouth getting into the industry. Oh, hey. You know, I noticed that you have these skills. I thought they're applicable, you know, applicable over here.

How do we how do we reach people, especially in, network engineering you know, I think that the the, software development is much more accessible for people overall.

What I've seen don't know if maybe this is just my anecdotal view, you know, after going to conferences, my entire career, I'm thinking maybe fifteen percent women in the network side. It's supposed to be, like, I think it's, like, thirty to forty on a software development side, which, hey, they're doing way better.

So I feel like I feel like I should, you know, I should work on that, help it out. Anna Clayborn at Packett Fabric, makes that, you know, suggestion to bridge because her point is let's market to the software engineering, which looks more like early computer science. Right? It's probably not gonna get four times the representation as we do early in career coming out of you know, school, but there are definitely things turn people of all sorts of diverse backgrounds off to, you know, a a group that's not like them.

But she says she suggests we market, hey, if you're interested in distributed systems, the internet is the biggest most dynamic distributed them there is. And if you wanna study it, well, that's what people I like doing, if you wanna engineer it, you know, and so and you know, the trick with automation, right, is not having forty engineers each write a Python program of their CLI thing, but you know, what are the patterns and let's, you know, think about it and do it. So Right. One system.

Not forty. Yes. Yeah. Have you tried that recruiting tactic yet?

I have not, which I think is a fantastic idea. Second second and see. Have to try it? Definitely.

Have have there been any other organizations that were as, had as many great women you know, and, you know, trusted in, you know, in in in authority, you know, as as what you experience in Digex, you know, later in your career?

I think Cogent also had a pretty good representation as well in the network engineering side in particular.

It it wasn't quite the fifty fifty. Like, we had, you know, at Digex.

You know, the and network engineering team at that time was, like, fifty fifty. It was, like, totally, like, it was wonderful.

I'd, you know, it but I you know, I think that they did have a pretty good representation.

I'm working on it.

We're in further behind and working on it. And, I mean, I think some of that that we think about, which is maybe tougher during COVID, maybe not.

Is earlier stage. Getting people earlier in stage or earlier in career because there's there is more diversity in you know, maybe showing the fun, but also worry a little bit about, you know, the tribal nature of it, which Also, different people do better or worse at, hey, I'm confused. Can you please help me? Or, you know, formalize that to make sure that I mean, it's not just gender.

It could be different cultures that people come from where, you know, some people are better at getting feedback or getting feedback or all these things. So, we will figure that all out together. Yeah. So you mentioned budget and digits.

Yeah. And Salesforce.

So it's all networking. Right? It's all never but service provider and having customers that are themselves service providers, although in some sense, I mean, we're a customer of Salesforce, we're a SaaS company So we are a customer of a disservice provider to our customers, but it is a little different. And I'm I'm just curious what you've found, you know, is is like sort of technology, you know, culture problems difference between working in service provider networks versus enterprise.

So I have worked at, I think it's three and three, actually.

So, three enterprises. I worked at a Financial Institute. I worked at just a pure enterprise and then, of course, Salesforce. Right?

And then, of course, the the ISP side, which I've actually had more time in. Right? So, it is vastly different. I find that, Some things are much more strict, the the very small enterprise that I worked at, I did everything, like, everything, like, you know, spam a stuff.

And I'm, like, running Linux machines. I'm helping database. I'm like doing firewalls and everything, which was great. All around nerd.

All around nerd. Yeah. Yeah. Exactly.

I loved it.

And then I think at, the, you know, cogent, the speed of innovation and the things that you get to do. Efficiency.

The efficiency of the network engineer for terabit. I It's amazing.

Yeah. The efficiency there is amazing.

Honestly, it's a fantastic network.

I I think that, you know, it it probably depends on the place, like, I feel like I probably don't get to do as much system stuff now as I did at cogent. Like, I could do more programming and systems work. Even being a manager, I could do all that stuff. Now it's a little bit harder for me to do that. You know, especially the team's bigger. Right? So I guess there's that dynamic up between the two, but, and I think PACE PACE is a bit different as well.

PACE is definitely different.

Do you work with more different vendors and sort of types of elements in an enterprise than, you know, in your experience, a service provider is sort of more you know, fixed and regular. It's fixed and regular. And, especially with cogent, when you're talking about cogent, It's it they know exactly what they're doing. They have set of products, and these are the products we sell. And it's very clear. I think I actually work with way more vendors right now, overall. It's just kind of fascinating.

And, I'm doing slightly different things in network engineering now. Obviously, there's differences in the network engineering side.

Does that make testing more complex when you have more vendors in different, you know, sort of case of newness your your customer, is not a product manager. It's a someone trying to make something. And the application's gonna do and be hosted the way that people want it to. Is that make testing, or is it you have patterns that would help with that? We do have patterns to help with that, but it is more the the testing is, I would say you wanna make sure that not only is the network running and everything sufficient, but that the applications are appropriately running over it. So you're working with lots of service owners within the organization.

And that's kinda fascinating too. It's kinda cool. It's like there are customers, you know, and we get to to work them. Hey, this is my customer.

How is your application running? You know, what do you need? That kind of stuff there's that kind of relationship, which I think is fascinating and kind of fun. Cool.

Yeah. I guess that another time, we can get into the how do all the groups, and we didn't talk about security. Like, again, infrastructure applications and all that. That's the that's sort of my day job, you know, thing and trying to figure that out.

And a lot of cultural divide there too. I mean, as as much as service provider and enterprise networking, and prod and corp networking can be, you know, different applications, different, you know, people we deal with different vendors, you know, going across groups can be a lot. But, you know, at at Salesforce, do you work with cloud as well as more traditional I'll call it on prem, you know, routers and switches and things like that. Is that We're starting to, on that front.

Yeah. Absolutely. But Mhmm. Miss Devin, is that interesting, difficult from architecture, provisioning, training, you know, you know, it's it's network principles underneath, but it isn't exactly the same.

Right? No. It it's Yeah. We are embarking upon this process of making everybody, making sure everybody is educated in how the different providers provide the the services, you know, and they're vastly different between the different, you know, cloud provider vendors Right?

So you have to make sure that, okay, well, you understand how this one works and understand traffic flows and how this one works. And, okay, you know, This one provides this, and this one provides that, and, you know, trying to get everybody in a level set. So there is a challenge always of making sure everybody is taking classes, you know, getting to do all the different types of work.

Yeah. I mean, it's interesting for us to a kentech.

Again, we know what tunnels are. Right. And we know what prefixes are. And we know what filters are.

And we know you know what all the basic primitives are if you've been paying attention to networking, but, You know, they just the infrastructure is code wave and and what people wanna do there. And then in cloud basically you're living in someone else's virtual software land, that, can be hard to do the debugging and, and, Yeah. The training is interesting. We're we're trying to do content, you know, internally and externally to help do that mapping.

You know, as a lot of people are trying to trying to do that, do you have people that are like deep experts in the ins and outs of cloud one or cloud two or do you sort of You mentioned cross training, like, have people be sort of enough per peripherally aware that they can be a consumer and then have, like, domain experts?

So, yes, we have, some people that are deeply involved in them. We also have just a series of wonderful classes that a team has help to make sure that, anybody who wants to can learn how each one Okay. Is done. And it's actually self case so the engineer can do, you know, do that. And then sometimes there's a project that comes up and we're like, hey, everybody, you know, Would you like to take this opportunity to go work on this new thing? So we try to make space for people to be able to do that as well. You know, in networking, there's a lot of vendors, a lot of marketing, a lot of buzzwords.

Like, are there, trends and buzzwords that you're tired of hearing of? And are there things that you think are this counter is, like, really fascinated by, and her think are, you know, really cool evolutions of technology that that are really fun or you're you think are really promising. Like, what's hot and what's hype?

Okay. Hype. Software defined.

Okay. I'm really sick of it. S t star s t star? Yes. Software to find anything. It's all software defined as far as I'm concerned.

The router isn't doesn't have marbles inside.

Right. Exactly.

So that whole thing, I was like, okay. Okay.

Delivery of information is certainly interesting to me, trying to figure how to better understand the user experience always fascinates me.

Because, you know, we don't want to, you know, we're very careful making sure we never give any customer information away, but we also are cognizant that we wanna make sure the customers experience as good as possible. So, that place that trying to find, like, what is going on with the customer, That's that's huge for me. And then the things that are underlying that, I know this is gonna sound awful. This is you're like, oh my god. KMZs.

You're like, oh, you're kidding me. Right? So Yeah. We take an enormous amount of time looking at every single path that we're using on the on the network.

Right? And we're spending so much time doing this, and then you know what's gonna have often is that they're gonna groom you out of that path as soon as they can, you're on the path. And you don't know the path. On the physics.

How does it go? Thirty five milliseconds between LA and San Diego, I guess.

Right. We spend all this time doing this stuff, and we're still doing the same things that we were, you know, like the last decade, or two.

So, yeah, I think that for me that trying to make the user experience is as amazing as possible, looking at all that data, how do I make it easier for my engineers to get to make it better for for everybody?

I know that's probably not the biggest hype, but, hey, it's where I'm at right now.

It it's part of my frustration because You know, at Akamai, we had the data.

When I run a username company, I I would like look at kernel TCP statistics and what AS it was going to and look at peering performance.

But, yeah, I mean, routers and switches, unless you're doing That's an advantage actually of IT networking has, like, some of those devices like a silver peak load balancer or some things like that can look at performance and they're sort of network planner. But, you know, routers and switches, Netflow and Nestflow. I mean, I've asked some of the vendors could you flow hash so we could look at TCP session tracking. Let's ignore quick for a second.

But, you know, but, yeah, I mean, the the rum data you would think would be really helpful for network people, but it has user data in it. So, like, how do you get that between layers and combine it and how do router vendors do this? And so what we're left with ping and trace route. Yeah.

It was like, how do we have trillions of endpoints? How do we have who can we partner with for that? Now that we're doing, you know, performance testing. But it sort of still feels like if our computers know what the performance is, why are we doing sympathetic testing?

You know? And it's like, it's an augment. Right? Sometimes you need that. Right? Like, having a problem to Fu in Barr, can Baz or Jill do better?

Like, you need to test that. But for a lot of it, it does seem like we could do better. But Yeah. Being against, like, cross stack.


APP you know, server infra.

So They're like all over the place. Yeah. I mean, we're hoping more people will do, like, BTF on the server, which I can take that with BGP and C performance from TCP stats, but now it's like, at Kentic, we're working with New Relic because you know, that helps our network team go to an app person and say, oh, you have the agent. Could you please do this? But it's still a teammate.

Sorry? How you go to my customers and say, could you help me out? Help me help you. Help me help you. Show me, show me their performance.

And, yeah, we'll see.

What I learned was overly advanced. If you're ahead of the market of people do or what people are familiar with, then people, you know, can't explain it to other people. Yeah. Instead ping and trace about people can explain, and we can make really easy and we can make it dynamic based on your traffic. So that's, you know, that's the focus.

But I think most network people are like that. You know, and of course, it's always blame the network. The network is the problem. Oh, it always is.

I'm sure not at Salesforce, but just in general, You know, it's the it's it's the, you know, it's it's it's the it's part of that.

And yeah, automation, what is can you you mentioned lab? So I guess, you know, how do you how do you lab stuff? And can you do that across all the infrastructure? Or you know, how do you do safe networking?

We also okay. So, We're getting into a little fuzzy area. I can give you generalizations. So we have different types of networks. I mean, we have, kind of like mini versions of them in the lab. And then we we use all kinds of different tools to make sure that when we are, you know, code certification, making sure all of that stuff works properly.

My team is now responsible for that as well. So I've been doing a lot of that.

Good times.

Making sure that when we make changes, we first check-in there. What does it look like when you make the change? What are the commands that you can use, making sure that everything in the run list when we do it is fine. We have, we have kind of like, pro not they're kinda like, you know, r and d environments that we can then canary we have DR environments where we can canary for certain types of networks.

So we and then we always canary and stagger and, you know, that kind of thing. So we try to use the best possible methodology for putting anything anywhere. So to break it down for people who might be looking at the outsider, I'm gonna guess. So by Carrie, you mean like a percentage of the traffic goes through some new infrastructure or So if, like, for instance, you're making a change to, you say you have to make you have to add a new route policy to the network.

Right? You take that route policy, you put it on one of the two routers in a location. So first, you would put it in r and d, and you would put one and two routers at a location, verify everything's okay, put it on the second router, still verify everything's okay, then you start doing regional canaries. Right?

So you You put, like, you put it on, you know, one router pair in one region and one router pair in another region. Okay. Then you can then kinda go out from there. So it's like every single step of the way you are being as careful as possible.

Awesome. Yeah. I mean, we did that at Ocamai And Yeah. It can take we have an hour from. So we actually copy, a bunch of customers worth of traffic to a separate cluster.

But we don't have real users on it. Ultimately, when we actually say, okay, the code is good, then, yeah, we do the same thing. We'll pick a cluster and then a machine. And I I'm not sure we do that for the networking.

So I'll have to go follow-up with our ops team. I think they might just roll the complaint, but it's all Juniper and Well, I would say, vendor and it's all a while back and, you know, like that, but, an automated, but, yeah, no, that's, that's cool. I always imagine, like, I have a friend who work who is a massive vendor torture. I I longed to be as good as member torture, but I always imagined, like, the Sandroid, you know, truck from Star Wars with the droids upside down with the hot Pokers, you know, like, does this code work?

Does this code work?

How do you do that, you know, people that I think I have for it even though we try to automate it.

You know, what features do we need and will it work? Yeah. In every in different environments, you know, require different features, and you have to make sure that all the different pieces are tested in every single different kind of, platform that you've got. And, yes.

Yes. We, you know, we are constantly making sure we're checking for vulnerabilities or checking for bugs when I go to, you know, like, okay, what's your, you know, what's your latest code you want me to try? Okay. Well, then we do the bug scrub, and you're like, You know, we try to be as nice about it as possible, though.

I'm not really into being mean about it. Okay. So new venture torture. I like that next rub.

Is that is that a industry term that I Yeah. Basically, we basically go and say, okay. So you wanna put us on this code. What bugs are there in the code?

And then we compare it to what the feature set on the network and see which ones match up. Okay. We're gonna hit this bug. Yeah.

We wanna wait for a different, you know, version of code.

And, of course, you sometimes catch bugs on your own. And we haven't, you know, we in the lab, it happens. You, you know, you're like, oh, look, or we we catch it and we have caught things in what we call the canary, the first step. You the very first thing, you're like, oh, look.

And then we will retroactively put it back in the lab and so, you know, yeah. That's funny in the wild West is when my API would be in a city and is like, oh, could you help me do dynamic routing from my static network? And they're like, What IGP should I use? I'm like, well, what cards do you need?

What vendors do you have? Let's look at the release notes. So it's called a bug scrub. It's like, you put that fiddy card in, then sparks spit out the hissing port.

If you have this protocol enabled, it's like, But, you know, literally there was a point at which I was, like, o s p f, even Rip V2 got help you. ISIS, like, just IBGP. Pick the thing that has the fewest bugs for your platform, but the flip side is that was without testing and deploy, and that was clickety clickety. So as you said, the human errors, can go.

Yeah. And, you know, we've seen we've seen some move towards pushing vendors to have, like, fewer trains, you know, and it's tough as a vendor because you might have an older, basically, program loader that was before page memory management units and preemptive multitasking and all that. You have the newer OS, but, you know, not everything. So, you know, it's tough. It is hard. I know. And, have some sympathy.

Yes. I know.

I would not, you know, when occasionally I go on my streaming telemetry rant about how it would be nice if, like, the thing that was the OID and the show int and the API and the CLI and the streamline telemetry all had the same semantics.

But that's often not the case. And so that's okay because then people pay us to fix that problem. But it's still, you know, someone at the end has to say, like, why did Why? What did some intern do to this thing?

But I think some of that is us as networker maybe ask more for features than we do. Not like you and me, obviously, or our peers, but some companies wind up like, hey, I need this feature it's enterprise and service provider and not, like, it must work work work work work work work work. So Yeah. I agree with you there. Yeah. That was fun. So We talked a little bit about your, you know, early career.

Are you hiring at Salesforce?

There are different groups that are hiring, absolutely, especially in our global network operations team.

So, yes, absolutely. So When people are looking for a job in networking, like, what are the things that you you know, that you look for, how can someone show their interest?

And you know, and sort of break into it. I mean, we thought it was a very different time when we broke into it. Like, what would your advice be, to people trying to get into networking?

Okay. So I personally, yes. It's it's great to know every last routing protocol on the planet.

What I am more interested in personally, can you learn? Are you inter or can you go and give I approved to me that you can go and actually learn something about one thing and apply that to a couple of different other, you know, you know, Say, I have a love of BGP.

Mhmm. I have then gone and decided that I wanna go learn about how BGP route reflection is used, and How does he use across these huge, large networks? And what are they doing? How are they doing it?

You know, where are they sitting at the route reflection? Are they using surfers? Are they not using surfers? Like, whatever it is.

Pick a thing. Learn about it. I wanna understand that you know it and that you can apply it to something else. I I don't care that you have, like, every single little detail of knowledge of everything. Can you trouble can you learn, you know, that kind of stuff? I'm not as, you know, you don't have to, like, tell me every last aspect of every, you know, vendor on the planet and what they support in every version of code for every line card, and I don't care.

So is there is there like an equivalent for developers?

You know, where they do things on GitHub to know that they're, like, active and interested. I guess network people can be GitHub too now with, you know, what I mean? We do.

Exactly. Oh, please don't test me on git. Please.

Submit clone.

Oh, yeah. Or hacker, you know, hacker or, you know, what is it Hacker ranked, that one? Is that the one that they're using? Yeah.

That one is a one. Yeah. That does the Yeah. I I would I mean, I still know, like, perforce and SCCS, and stuff like that.

So, from before before the great getting, you know, unveiled upon the place.

Is that, you know, also something like that you look for people that already have had some, you know, at least scripting, you know, level coding? Yeah. I mean, we we do frequently ask that, you know, if you've had even Pearl or Shell or Python, At least that you're interested in learning these things, that's important because, you know, we we use it. And even, you know, in self defense, and, you know, for making things quicker for yourself, it's helpful.

Right. As long as you configure it so it's quicker to do good things, not quicker to break things. Yes. Yes. But even for myself, on my machine, like, there are times where I just write something really quick to, like, do the thing for me so that I don't have to, like, do it by hand.

Yeah. I on the more assisted men's side, I ran, you know, a using that company for ten years, as a hobby. And, I I would sometimes find that I had written the same Pearl script five times, which is why I'm not a software engineer. I'm more like a irate running specification.

You know, it's like, hey, I could do this.

It's like, okay. Now, please, people that I've hired to do great work, you know, make it all That is honestly precisely what I did. Many times in my career.

Just I've got everything. I know what it has to do. This is all the data that you need in and out, and please make it so that it's not horrible. Thanks. It's more precise than English. So even if it's not good code.

And hopefully, we write pro code that looks like c, not like but, you know, it's possible to do evil with even acceptable things. So, I guess this is a strange question because I don't have a degree, but, you know, is is a degree important are there things that people can do if they don't have a degree to, you know, show, you know, interest or education or awareness?

Sure. So I don't feel like degrees are particularly important because I didn't ever finish either. Neither did my husband. So it really seems to be doing just fine.

There are ways that you can do this. There are different vendors who have their own testing and that kind of thing. And, you know, I Absolutely. Those are helpful.

Unfortunately, sometimes those are just pure memorization, and I do worry about that. We did have some successful, mini labs set up at one company.

And they, you know, went through some troubleshooting steps, and that's actually really helpful.

You can kinda do, like, a lab or you, like, shut down a port and you see, like, how the person reacts to, like, oh, hey. You know, your routes for this went away. And, you know, what do they notice? Did they look in this this log? Did they, like, that kind of stuff? So I think those are really excellent methods.

We've had, you know, obviously there's, you know, text screenings and you go through, like, some questions? How did they answer? What were their answers?

You know, some things where they get the concepts and miss some of the details.

You know, more okay with personally.

Yeah, the concept is How does someone learning? And, you know, yeah, it's like You almost wanna make sure that people are comfortable, comfortable getting confused and owning the Unconfusion versus, you know, sort of training in a class, but you have to provide the resources to make sense. Right? Yeah.

Like, I I feel like getting confused is excellent. I'm confused all the time, and I feel like it's it's okay. I mean, I have to ask, you know, like, I don't have perfect knowledge of everything happening. Some I have to ask questions all the time.

And I think that very thing actually within teams is helpful.

I'm not gonna know, like, all four year, fifty projects that are going on right now. Right. That's okay. Any vendors, but I I like to say networking is a lot of simple little things that interact in complex ways, and then there's vendor bugs.

And you're unlikely to find a kernel bug, maybe in your career, but it depends how low level, you know, you have to you always have to say, well, wait, is this a bug? So you have to run through, like, is think doing what it should be? Do I understand what it should be doing? So it's, I also I also think that a lot of the vendors have really good virtual routers and stuff Right now, you can set up your own, like, little mini network, you know, tech Yeah.

VMX, VSRX, you know, There's that has on, you know, online and Now there is. Yeah. So, we study, in the physics of computer stuff. There's bore bugs, which always are the same, and there's heisen bugs which move around.

And heisenbergs move around when they're into buggers, right, because the stack is different. But I've seen a couple of cases where the physical, the software not running on the router, but running, or the switch, but running in a VM as different bugs. That could be fun. Oh, no.

No. No. And, and, of course, you can't actually do everything that you might wanna test and getting tervets, go forwarding and Well, yeah. And and even just some of the features don't work because of the way things line up.

So for instance, certain lag features or Don't really work properly.


Cool. Exactly.

Well, Thank you for, sharing so much. Networking can sometimes frustrate us, but I think it's fun to see all the packets go and all that. I guess last question. Any advice you would, give yourself going back, you know, navigating, you know, for career or otherwise? Yes.

As a woman in the industry, it has sometimes been a bit challenging dealing with a lot of forceful personalities, it's okay to be a forceful personality. It's okay to meet that where it's at.

You know, I think that that sometimes, it it is a challenge because you kinda feel like you're being attacked. And I think that it's okay to be that forceful personality and and, you know, not just, you know, kinda like, shrinking yourself, which is, I think, a challenge for everybody.

I mean, I've seen people do things, men do things that really are not respectful, and you try to affects and help if you can. And then I have seen separately in networking people that get so ashin' about the geek binary hiatus, the way to do it. Somebody on the internet is wrong. Yes. Exactly. And you said, but you know, the it's not that I hate you. It's that I hate the idea, and I will now prove to you that it is wrong.

Is that you know, or I guess, you know, so so it can be heard, and I guess we owe it to, mean, whoever it is women or just people in in Anybody? Try to help. Yeah. Do that.

But I guess related. I said that was the last question. So that was the second last question. Last question.

What can we do to help people from different backgrounds of all sorts, to make networking that, you know, people wise, you know, more acceptable given there are people on all ends of all spectrums, you know, in it, but that get very passionate and, you know, whatever. What what advice would you have for your org? My org, you know, people in general, you know, trying to bring people in and then set them up for success, you know, to have career, you know, not just I do I tell you to do these things and you do a job. Yeah.

Yeah. No. I I think that, one thing I try to do with my teams, and I I hope that I'm successful in this is that we talk about all kinds of aspects of the job. Like, we try to bring out, like, okay.

What's going on here? What are the blockers? What can we do?

Oh, you know, I saw that your face was you seem to be upset about this. Or do you feel comfortable talking about it in the team? You know, okay. I have this project. I have a really strong opinion on this, what should happen.

But I don't actually say my opinion generally until everybody else has gotten their their opinion out because what I don't wanna do is I don't wanna I don't wanna, like, stifle the conversation that's happening. But, you know, we all do try to keep it as respectful as possible.

You know, and I have also coached some people that sometimes you need to just, not push people because they're not comfortable.

You know, so there's you have to kinda balance everybody out. Everybody's got a a different place they're coming from. So you just have to try to keep the information flowing, get everybody talking. I I try to I hope that it's more like a family environment than my teams. That's what I try to aim for.

Yeah. It's it sounds very cool.

It it it is difficult. I mean, even for me, I mean, for people that have opinions, you know, because we're technologists or organizational not psychologists, but, you know, we have opinions about things to the to, like, in theory, I wanna say, you know, if you need an opinion, I have one, but I will only, you know, I will tell you, but I'm only trying to correct if it's, like, really wrong.

It can be tough. And then No. No. No. And that's exactly one thing I try to do some on some particular things, right, where I'm like, okay.

Have a strong one. I have a strong feeling on this one, like, especially with routing things. I'm like, okay. Just gonna put a hold back.

Yeah. Or SD, you know, it's like Well, that one too. Ultimately, the most successful SD thing out there is probably NSX. Which started as I hate network people, so we're gonna build tunnels because we don't wanna put in a ticket to get a VLAN. And so, you know, in some sense, I think it's okay to you know, plus Google got everyone spun up on it. Google's a software company.

So, you know, it's, like, if you wanna write software, then you know, yeah, SD is gonna look pretty good, but, anyway, well, cool.

How can people, find you, you know, do you have a blog, Twitter face, not Facebook, that's private?

You know, the funny thing is is I don't actually I'm so sure about this. I'm so bad at this.

I, you know, LinkedIn is probably a good one. I'm on LinkedIn.

That's probably the best one. I don't really have, like, I don't have a, like, I my everything's personal. Like, I just don't I don't talk about work.

That's okay.

Gail, is my wife is not on Facebook. So her high school, classmates are like, you know, they send me something on Facebook if I should send it to her. She just refuses to buy in. To the ecosystem. But That's funny. So my husband, Joe, as as you, I think you probably know I'm, very similar on that front. So I'm the I'm the person who does everything.

Okay. Yep. That's fine.

And, but anyway, thank you for being the second guest on network and helping me evolve this. And, we welcome all into, networking and hope this was helpful. This is great. I really appreciate it. It was a lot of fun.

Got a guest?

Network AF is accepting guests for upcoming episodes. If you’d like to be on the podcast or refer a friend, reach out to

About Network AF

Network AF is a journey of super-nerd proportions into the world of networking, cloud, and the internet. Avi Freedman, self-described internet plumber and podcast namesake, hosts top network engineering experts from around the world for in-depth, honest, and freewheeling banter on all-things-network — how-tos, best practices, biggest mistakes, war stories, hot takes, rants, and more.
We use cookies to deliver our services.
By using our website, you agree to the use of cookies as described in our Privacy Policy.