If you write
pass in on ep1 from ep1:network to ep0:network \ port $ports keep state
then you also need
pass out on ep0 from ep1:network to ep0:network \ port $ports keep state
but do you actually mean
pass from ep1:network to any port $ports keep state