The account does not have permission to impersonate the requested user

I get a runtime error trying to impresonate a user - code below

  Service = New ExchangeService()

        Service.Url = New Uri(ApplicationItems.ExchangeURL)
        Service.Credentials = New NetworkCredential(username, password)
        Service.ImpersonatedUserId = New ImpersonatedUserId(ConnectingIdType.SmtpAddress, emailAddress)