Hexadecimal to Decimal

Post

Posted
Rating:
#1 (In Topic #585)
Avatar
Guru
cogier is in the usergroup ‘Guru’
Print &HFFF Returns 4095
Print &HFFFF Returns -1 not 65535
Print &HFFFFF Returns 1048575

Am I missing something here?
Online now: No Back to the top

Post

Posted
Rating:
#2
Avatar
Regular
stevedee is in the usergroup ‘Regular’

cogier said

Print &HFFF Returns 4095
Print &HFFFF Returns -1 not 65535
Print &HFFFFF Returns 1048575

Am I missing something here?

Try:-
Print &H00FFFF  (Returns  65535)
Online now: No Back to the top

Post

Posted
Rating:
#3
Avatar
Enthusiast
PJBlack is in the usergroup ‘Enthusiast’
Byte    0 … 255        1 Byte
Short    -32.768 … +32.767        2 Bytes
Integer    -2.147.483.648 … +2.147.483.647        4 Bytes
Long    -9.223.372.036.854.775.808 … +9.223.372.036.854.775.807     8 Bytes

Print &HFFF nearest fitting data type = short -> Returns 4095
Print &HFFFF nearest fitting data type = short -> Returns -1 (which is correct)
Print &HFFFFF nearest fitting data type = integer -> Returns 1048575
Online now: No Back to the top

Post

Posted
Rating:
#4
Avatar
Regular
stevedee is in the usergroup ‘Regular’
 Yes, I think its a casting problem. Adding a trailing "&" seems to sort it out:-
&hffff&
Online now: No Back to the top

Post

Posted
Rating:
#5
Avatar
Guru
cogier is in the usergroup ‘Guru’
 Thanks, that extra '00' works but why?
Online now: No Back to the top
1 guest and 0 members have just viewed this.