Oh man, that would've made for an interesting conversation then. Sam would be like "oh hey Loki, glad you're here, have some ice cream." He'd be totally ok with Loki not remembering him, too. They can just make new memories or sth.