Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claims 1, 10, and 16 have been amended; claims 5 and 6 have been canceled; claims 1-4 and 7-19 are pending.
Response to Arguments
Applicant’s arguments, see pages 1-3, filed 11/19/2025, with respect to the rejection(s) of claim(s) 1-4,6-19 under U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made, see rejection below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-4, 10-13, 16, 17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Lin (Pub 20120044421) in view of Lu (Pub 20240305754) in view Huh (Pub 20200242799) in view of Gibson (Pub 20090147272) in view of Furuichi (Pub 20120092630).
Regarding claim 1, Lin discloses:
a projector that is configured to project modified video onto a surface in an image field, (212 and 216 fig 2);
a camera having a camera view which includes the image field of the projector and that produces image data of the camera view, (Para. [0019-0020]);
an image processor coupled to the projector and the camera, that recognizes an object in the image data that is in the image field of the projector between the surface and the projector which generates a mask that is applied to a source video to create the modified video that is projected by the projector, wherein the projector changes light being projected in a region of the mask in the modified video, (image processor 206 fig 2 coupled to 210, 212, and 216; which generates mask based on detected objects 410 and 414 fig 4 and Para. [0021][0030][0033-0035])
wherein the mask is generated using the at least one object recognition model to determine spatial data of the object in the camera view and by applying a correction to the spatial data, (see image recognition Para. [0031] and correction in Para. [0021][0024-0026]).
However, a camera that is spatially offset from the projector having a camera view which includes the image field of the projector at the surface as well as a space between the camera and the surface and that produces image data of the camera view, and wherein the correction is based on determining a portion of the object in the camera view that is in the image field of the projector based on the spatial offset between the camera and the projector and a position, between the surface and the projector, of the portion of the object in the image field of the projector are not disclosed
In a similar field of endeavor, Lu discloses a camera that is spatially offset from the projector having a camera view which includes the image field of the projector at the surface as well as a space between the camera and the surface and that produces image data of the camera view, (see at least camera 251 fig 9/fig 12/ fig 15 where camera 251 is spatially offset “horizontally” from projector 230); and
wherein the correction is based on determining a portion of the object in the camera view that is in the image field of the projector based on the spatial offset between the camera and the projector and a position, between the surface and the projector, of the portion of the object in the image field of the projector, (note camera 251 fig 9/fig 12/fig 15 is spatially offset from projector 230 fig 9/fig 12/ fig 15. Also note fig 25 disclosing keystone correction and obstacle avoidance comprising steps correcting images using relationship between camera coordinates and projector coordinate to account for camera-projector spatial offset Para. [0212] and determining a portion of an object “blocking object” in camera view 251, see Fig 9/fig 12/fig 15 as well as determining position between the surface and the projector of the portion of the object in the image field of the projector, (Para. [0217-0224]). One of ordinary skill in the art would include the correction and obstacle avoidance of Lu in the combination so that harmfully rays are inhibited from reaching vision of a person present or passing by between a projector and a projection surface. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination by incorporating Lu for the common purpose of allowing image correction to be implemented and enhancing the final projected image.
However, a database in which a plurality of object recognition models is stored and an image processor that recognizes an object in the image data that is in the image field of the projector by applying the image date to at least one of the object recognition models is not disclosed.
In a similar field of endeavor, Huh discloses a database in which a plurality of object recognition models is stored and an image processor that recognizes an object in the image data that is in the image field of the projector by applying the image data to at least one of the object recognition models, (Para. [0042])). One of ordinary skill in the art would have included the object recognition model of Huh in Lin after the image is captured. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Lin by incorporating Huh for the common purpose of allowing objects to be recognized faster thereby improving overall projector performance.
The combination does not disclose measuring object distance. However, this is known and conventional. For example, Gibson is of a similar field of endeavor disclosing measuring or estimating a distance from a projector to an object, (Para. [0030] see distance remeasured. Further note based on distance either a proximity mode or a normal mode is executed). It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination by Gibson so that when an obstacle is present, light is adjusted according to how far an obstacle is from a projector thereby protecting light from reaching the obstacle.
The combination does not disclose wherein the correction is adjusted as the object moves closer to or father from the projector while remaining in the image field of the projector.
In a similar field of endeavor, Furuichi discloses wherein the correction is adjusted as the object moves closer to or father from the projector while remaining in the image field of the projector, (see adjusting light amount of projection devices according to different zones occupied by an object or person, see Fig 6-8. In Fig 6, human sensor for determining how far a person is from a projector within an image field is decided in S2, and based on an outcome of human sensor at S7, a determination is made whether to switch to normal light amount projection S8 or not S6. In fig 7-8, based on setup 2, the amount of light is adjusted when a person is in area A to a minimum when human sensor output is LO whereas the light is adjusted to be normal when a person is detected in area C when a human sensor is HI. In addition, note that a person moving closer or farther can be determined in S6-S7 Fig 6 construed as a person being closer to a projector and S7-S2 construed as a person moving farther from the projector requiring normal mode of operation). It is further noted that the amount of light may be maintained even when a person is detected at a farther distance, Para. [0015]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination by incorporating Furuichi so that light amount is adjusted depending on how far a person is from projected light source thereby projected images are clearly displayed which enhances a user's enjoyment.
Regarding claim 2, Lin discloses wherein the projector changes the light being projected in a region of the mask by reducing a brightness of the light being projected in the region of the mask relative to what would have been projected using the source video, (Para. [0035] dimming).
Regarding claim 3, Lin discloses wherein the light is blacked out in the region of the mask, (Para. [0035] black out).
Regarding claim 4, Lin discloses wherein the projector changes the light being projected in a region of the mask by projecting a graphic overlay in the region of the mask instead of the portion of the source video that corresponds to the location of the mask in the image field, (creating a blank masking image 412 to be combined with video-in signal Para. [0030]).
Regarding claim 10, Lin discloses:
a projector operable to project a video in an image field, (212 and 216 fig 2);
a sensor operable to generate an image that includes the projected image field of the projector, (220 fig 2);
an image processor responsive to the sensor that is configured to process the image view generated by the sensor and detect an object in the image view of the projector, (206 fig 2 Para. [0020]),
and generate a mask corresponding to the object in the image field of the projector, wherein the mask is applied to a video source to produce a modified video source that is projected by the projector wherein light in the region of the mask is modified from that of the video source, (image processor 206 fig 2 coupled to 210, 212, and 216; which generates mask based on detected objects 410 and 414 fig 4 and Para. [0021][0030][0033-0035])
wherein the mask is generated using the at least one object recognition model to determine spatial data of the object in the camera view and by applying a correction to the spatial data, wherein the correction is based on determining a portion of the object and a position of the portion of the object in the image field of the projector, (see image recognition Para. [0031] and correction in Para. [0021][0024-0026]).
However, a sensor that is spatially offset from the projector that is operable to generate an image view that includes the projected image field of the projector at the surface and a space between the sensor and the surface; and wherein the correction is based on determining the portion of the object in the image field of the projector based on the spatial offset between the camera and the projector and a position, between the surface and the projector, of the portion of the object in the image field of the projector are not disclosed.
In a similar field of endeavor, Lu discloses a sensor that is spatially offset from the projector that is operable to generate an image view that includes the projected image field of the projector at the surface and a space between the sensor and the surface, (see at least camera 251 fig 9/fig 12/ fig 15 which is offset horizontally from projection lens); and
wherein the correction is based on determining the portion of the object in the image field of the projector based on the spatial offset between the camera and the projector and a position, between the surface and the projector, of the portion of the object in the image field of the projector, (note camera 251 fig 9/fig 12/fig 15 is spatially offset from projector 230 fig 9/fig 12/ fig 15. Also note fig 25 disclosing keystone correction and obstacle avoidance comprising steps correcting images using relationship between camera coordinates and projector coordinate to account for camera-projector spatial offset Para. [0212] and determining a portion of an object “blocking object” in camera view 251, see Fig 9/fig 12/fig 15 as well as determining position between the surface and the projector of the portion of the object in the image field of the projector, (Para. [0217-0224]). One of ordinary skill in the art would include the correction and obstacle avoidance of Lu in the combination so that harmfully rays are inhibited from reaching vision of a person present or passing by between a projector and a projection surface. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination by incorporating Lu for the common purpose of allowing image correction to be implemented and enhancing the final projected image.
However, a database in which a plurality of object recognition models is stored and an image processor that recognizes an object in the image data that is in the image field of the projector by applying the image date to at least one of the object recognition models is not disclosed.
In a similar field of endeavor, Huh discloses a database in which a plurality of object recognition models is stored and an image processor that recognizes an object in the image data that is in the image field of the projector by applying the image data to at least one of the object recognition models, (Para. [0042])). One of ordinary skill in the art would have included the object recognition model of Huh in Lin after the image is captured. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Lin by incorporating Huh for the common purpose of allowing objects to be recognized faster thereby improving overall projector performance.
The combination does not disclose measuring object distance. However, this is known and conventional. For example, Gibson is of a similar field of endeavor disclosing measuring or estimating a distance from a projector to an object, (Para. [0030] see distance remeasured. Further note based on distance either a proximity mode or a normal mode is executed). It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination by Gibson so that when an obstacle is present, light is adjusted according to how far an obstacle is from a projector thereby protecting light from reaching the obstacle.
The combination does not disclose wherein the correction is adjusted as the object moves closer to or father from the projector while remaining in the image field of the projector.
In a similar field of endeavor, Furuichi discloses wherein the correction is adjusted as the object moves closer to or father from the projector while remaining in the image field of the projector, (see adjusting light amount of projection devices according to different zones occupied by an object or person, see Fig 6-8. In Fig 6, human sensor for determining how far a person is from a projector within an image field is decided in S2, and based on an outcome of human sensor at S7, a determination is made whether to switch to normal light amount projection S8 or not S6. In fig 7-8, based on setup 2, the amount of light is adjusted when a person is in area A to a minimum when human sensor output is LO whereas the light is adjusted to be normal when a person is detected in area C when a human sensor is HI. In addition, note that a person moving closer or farther can be determined in S6-S7 Fig 6 construed as a person being closer to a projector and S7-S2 construed as a person moving farther from the projector requiring normal mode of operation). It is further noted that the amount of light may be maintained even when a person is detected at a farther distance, Para. [0015]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination by incorporating Furuichi so that light amount is adjusted depending on how far a person is from projected light source thereby projected images are clearly displayed which enhances a user's enjoyment.
Regarding claim 11, Lin discloses wherein the sensor is at least one camera, (220 fig 2).
Regarding claim 12, Lin discloses wherein the projector and the image processor are integrated into a housing, (image processor 206 and projector 212/216 fig 2).
Regarding claim 13, Lin discloses wherein the sensor is integrated into a housing, (Para. [0019]).
Regarding claim 16, Lin discloses:
Identifying, using a camera, an image field of a projector that is projecting a video in a camera view of the camera, (Para. [0028] alignment with respect to screen position and 220 fig 2);
detecting an object in the image field of the projector by capturing an image of the image field in the camera view using the camera, (Para. [0029])
determining a location of the object in the image field of the projector, (Para. [0030] pixels corresponding to object identified);
generating a mask that corresponds to the location of the object in the image field, (blank masking image created 412 Para. [0030]);
applying the mask to a source video to produce a modified source video in which content in the region of the mask is modified and projecting the modified source video, (image processor 206 fig 2 coupled to 210, 212, and 216; which generates mask based on detected objects 410 and 414 fig 4 and applying the mask Para. [0021][0030][0033-0035])
wherein the mask is generated using the at least one object recognition model to determine spatial data of the object in the camera view and by applying a correction to the spatial data, wherein the correction is based on determining a portion of the object and a position of the portion of the object in the image field of the projector, (see image recognition Para. [0031] and correction in Para. [0021][0024-0026]).
However, Identifying, using a camera, an image field of a projector that is projecting a video in a camera view of the camera wherein the camera is spatially offset from the projector; and determining the location of the portion of the object in the image field, wherein the correction accounts for the spatial offset as well as the position of the object in space relative to the projector are not disclosed.
In a similar field of endeavor, Lu discloses Identifying, using a camera, an image field of a projector that is projecting a video in a camera view of the camera wherein the camera is spatially offset from the projector, (see horizontal offset of at least camera 251 fig 9/fig 12/ fig 15 capturing); and
determining the location of the portion of the object in the image field, wherein the correction accounts for the spatial offset as well as the position of the object in space relative to the projector, (Para. [0217-0224]). One of ordinary skill in the art would include the correction and obstacle avoidance of Lu in the combination so that harmfully rays are inhibited from reaching vision of a person present or passing by between a projector and a projection surface. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination by incorporating Lu for the common purpose of allowing image correction to be implemented and enhancing the final projected image.
However, applying the image to at least one of a plurality of object recognition models is not disclosed.
In a similar field of endeavor, Huh discloses applying the image to at least one of a plurality of object recognition models, (Para. [0042])). One of ordinary skill in the art would have included the object recognition model of Huh in Lin after the image is captured. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Lin by incorporating Huh for the common purpose of allowing objects to be recognized faster thereby improving overall projector performance.
The combination does not disclose measuring object distance. However, this is known and conventional. For example, Gibson is of a similar field of endeavor disclosing measuring or estimating a distance from a projector to an object, (Para. [0030] see distance remeasured. Further note based on distance either a proximity mode or a normal mode is executed). It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination by Gibson so that when an obstacle is present, light is adjusted according to how far an obstacle is from a projector thereby protecting light from reaching the obstacle.
The combination does not disclose wherein the correction is adjusted as the object moves closer to or father from the projector while remaining in the image field of the projector.
In a similar field of endeavor, Furuichi discloses wherein the correction is adjusted as the object moves closer to or father from the projector while remaining in the image field of the projector, (see adjusting light amount of projection devices according to different zones occupied by an object or person, see Fig 6-8. In Fig 6, human sensor for determining how far a person is from a projector within an image field is decided in S2, and based on an outcome of human sensor at S7, a determination is made whether to switch to normal light amount projection S8 or not S6. In fig 7-8, based on setup 2, the amount of light is adjusted when a person is in area A to a minimum when human sensor output is LO whereas the light is adjusted to be normal when a person is detected in area C when a human sensor is HI. In addition, note that a person moving closer or farther can be determined in S6-S7 Fig 6 construed as a person being closer to a projector and S7-S2 construed as a person moving farther from the projector requiring normal mode of operation). It is further noted that the amount of light may be maintained even when a person is detected at a farther distance, Para. [0015]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination by incorporating Furuichi so that light amount is adjusted depending on how far a person is from projected light source thereby projected images are clearly displayed which enhances a user's enjoyment.
Regarding claim 17, Lin discloses wherein detecting an object in the image field comprises detecting the object in an image taken by a camera that includes the image field and recognizing the object in the image field in the image taken by the camera, (Para. [0020]).
Regarding claim 19, Lin discloses wherein the image processor is further configured to buffer the mask to be applied to the next frame of the source video, (Para. [0022] buffer 222).
Claim(s) 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Lin in view of Lu in view of Huh in view of Gibson in view of Furuichi in view of Nakamura (Pub 20220021856).
Regarding claim 7, the combination discloses a camera. However, an infrared
camera is not explicitly disclosed.
In a similar field of endeavor, Nakamura discloses wherein the at least one camera includes an infrared camera, (40 fig 1). One of ordinary skill in the art would include the infrared camera of Nakamura in the combination when capturing images of obstacles between a projector and a projection screen. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination by incorporating Nakamura for the benefit of capturing images without harming vision when an obstacle is person.
Regarding claim 14, the combination discloses a camera integrated with projector housing, (Para. [0019] Lin). However, a cable is not explicitly disclosed.
In a similar field of endeavor, Nakamura discloses wherein the sensor is connected to the housing via a cable, (Para. [(0041]). One of ordinary skill in the art may include the cable of Nakamura in the combination for connecting a camera to a projector housing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination by incorporating Nakamura for the benefit of reducing signal noise when multiple devices are present.
Claim(s) 8, 9, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Lin in view of Lu in view of Huh in view of Gibson in view of Furuichi in view of Morrison (Pub 20070273842).
Regarding claim 8, the combination discloses a camera, see claim 1. However,
wherein the at least one camera includes a time-of-flight camera is not disclosed.
In a similar field of endeavor, Morrison discloses wherein the at least one camera
includes a time-of-flight camera, (Para. [0090-0092] range finder emitting light and receiving return light)., (Para. [0090-0092] range finder emitting light and receiving return light). One of ordinary skill in the art may include a time-of-flight camera in the combination in addition to capturing images. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination by incorporating Morrison for the common purpose of providing additional means of detecting objects to enhance accuracy of determining object location.
Regarding claim 9, the combination discloses claim 1. However, further comprising a non-light sensor that transmits a signal and senses a return signal that indicates a location of the object is not disclosed
In a similar field of endeavor, Morrison discloses further comprising a non-light sensor that transmits a signal and senses a return signal that indicates a location of the object (Para. [0090] sound or radio wave range finder). One of ordinary skill in the art may include a non-light sensor in the combination in addition to capturing images. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination by incorporating Morrison for the common purpose of providing additional means of detecting objects to enhance accuracy of determining object location.
Regarding claim 15, the combination discloses claim 10. However, wherein the sensor includes a non-light sensor is not disclosed.
In a similar field of endeavor, Morrison discloses wherein the sensor includes a non-light sensor, (Para. [0090] sound or radio wave range finder). One of ordinary skill in the art may include a non-light sensor in the combination in addition to capturing images. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination by incorporating Morrison for the common purpose of providing additional means of detecting objects to enhance accuracy of determining object location.
Claim(s) 18 is rejected under 35 U.S.C. 103 as being unpatentable over Lin in view of Lu in view of Huh in view of Gibson in view of Furuichi in view of Agostinelli (U.S. 6984039).
Regarding claim 18, Lin discloses generating a mask per claim 1. However, wherein the image processor is further configured to feed the spatial data of the object back to a recognition process to use the spatial data in a processing a next frame of image data is not disclosed.
In a similar field of endeavor, Agostinelli discloses wherein the image processor is further configured to feed the spatial data of the object back to a recognition process to use the spatial data in a processing a next frame of image data, (col. 5 lines 55 57, see blanking lines from being scanned in next scan operation, i.e. subsequent frame). One of ordinary skill in the art would have included the features of Agostinelli in the masking operation of Lin. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Lin by Agostinelli for the benefit of reducing processing overhead by allowing the blanking operation to continue since an obstacle is likely to be present for longer time.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUMAM M SATTI whose telephone number is (571)270-1709. The examiner can normally be reached Mon-Fri.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Miller can be reached at (571)272-7353. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
HUMAM M. SATTI
Examiner
Art Unit 2422
/JOHN W MILLER/Supervisory Patent Examiner, Art Unit 2422