Roaming profiles and RemoteApp profiles

Here's a very strange behavior that my users are experiencing.

Our environment:

1. Windows 7 Enterprise x64
2. Local domain controller at the branch (Windows 2008 R2 Enterprise)
3. Users use RemoteApp to access JD Edwards. We have 2 clustered RemoteApp servers
4. All our users have folder redirection and roaming profiles enabled via GPO

Issue:

The issue seems to happen only with users that need access to our RemoteApp servers. Basically, after they've been logged on to our RemoteApp servers (usually most of the day, because they have to use JD Edwards), when they log off for the day, their profile changes and resembles the profile they have on the remote servers. In other words, their windows roaming profile now reflects the remote desktop server's profile (the powershell icon and the server manager icon appear on the users' task bar).

Is there a way to eliminate this behavior?