Fix memory leaks with weakref module #3

Job	Run time
opentelemetry-api	42s
opentelemetry-getting-started	1m 42s
opentelemetry-proto	29s
opentelemetry-sdk	31s
opentelemetry-exporter-opencensus	47s
opentelemetry-exporter-otlp-combined	48s
opentelemetry-exporter-otlp-proto-common	46s
opentelemetry-exporter-otlp-proto-grpc	1m 7s
opentelemetry-exporter-otlp-proto-http	49s
opentelemetry-exporter-prometheus	41s
opentelemetry-exporter-zipkin-combined	43s
opentelemetry-exporter-zipkin-json	37s
opentelemetry-exporter-zipkin-proto-http	44s
opentelemetry-opencensus-shim	39s
opentelemetry-opentracing-shim	48s
opentelemetry-propagator-b3	35s
opentelemetry-propagator-jaeger	36s
opentelemetry-semantic-conventions	31s
opentelemetry-test-utils	38s
	14m 13s

Provide feedback