Compare folder list to AD user list

I'm trying to clean up our home folder shares and want to compare current users from AD with a folder list from the file server. I want to take each variable and compare line-by-line... so say line 1 in $adusers is "joe.black", but "joe.black" doesn't show up until line 6 in $folders; how do I do that comparison? I'm thinking I'd start with this...

$adusers = get-aduser -filter {name -like "*"} | sort-object samaccountname | select-object @{Name='Name'; Expression={$_.samaccountname}}

$folders = get-childitem -Path \\fileserver\home$ -Directory | Sort-Object name | Select-Object @{Name='Name'; Expression={$_.name}}

I can't get compare-object to work because it's wanting to do apples-to-apples; I'm needing apples-to-oranges, in a sense. I'm thinking I'd need to do some sort of nested IF/ELSE, but am a bit fuzzy on it.