DETAILED ACTION
Allowable Subject Matter
Claims 2-3, 10-11 and 19-20 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 6, 8-9, 14-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kurokawa US 2022/0417483 hereinfter referred to as Kurokawa in view of Gruen et al. US 2011/0261260 hereinafter referred to as Gruen.
In regards to claim 1, Kurokawa teaches:
"A control method of an electronic device, the method comprising: adding a first pattern to a source image to obtain a first image"
Kurokawa Figure 5 and paragraph [0047] teaches The image processing unit 202 processes the image to be projected and output from the projection unit 201. The image output from the image processing unit 202 is an input image input from the image input unit 203 and a test pattern held in the image processing unit 102.
"projecting the first image onto a projection surface"
Kurokawa paragraph [0049] teaches The projection unit 201 projects an image output from the image processing unit 202 on a screen (not shown in FIG. 2).
"acquiring a second image of the first pattern included in the projected first image”
Kurokawa paragraph [0050] teaches The camera 103 captures a projection image in which a test pattern projected on the screen from the projection unit 201 is embedded during the projection status checking operation (or when acquiring the corresponding point information).
"based on the first pattern included in the second image, identifying a pattern region distorted [on the projection surface]"
Kurokawa paragraph [0051] teaches the corresponding point information acquisition unit 204 detects a test pattern from a captured image in which a test pattern is embedded, obtains information on a corresponding point between the original projection image and the captured image, and supplies the corresponding point information to the image processing unit 202. In the image processing unit 202, the correction amount for correcting the geometric distortion included in the projection image is calculated based on the corresponding point information, and the geometric distortion is corrected by performing the projective conversion on the output image.
"and changing a color or brightness of a portion of the source image that corresponds to the distorted pattern region”
Kurokawa paragraph [0057] teaches The image correction unit 403 corrects the image read from the frame memory 402 based on the corresponding point information received from the corresponding point information acquisition unit 204 so that the geometric distortion is eliminated when the image is projected from the projection unit 201 on the subject. The geometric correction has been described with reference to FIGS. 16 and 17, but is not limited thereto. Kurokawa paragraph [0058] teaches The image quality adjustment unit 404 adjusts the image quality such as luminance, contrast, synchronization, tracking, color depth, and hue so that the projection image after distortion correction is in a desired display state. The Examiner interprets and adjustment to hue as an adjustment to color.
Kurokawa does not explicitly teach:
"[distorted] by an object in front of the projection surface"
Gruen paragraph [0019] teaches The presenter 114 is positioned between the projector 104 and the projection screen 110, thereby occluding a portion of the projection of the first image 214A. As a result, the first projected image 216A contains a shadow 220 of the presenter 114. Gruen paragraph [0019] teaches the captured image 212A contains the projected image displayed on the projection screen 110 as well as any occluding objects (i.e., the presenter 114) between the projector 104 and the projection screen 110. The camera 204 provides the first captured image 212A to the computing device 202. Gruen paragraph [0021] teaches The image enhancement module 206 generates the new second image 218 by modifying the pixels identified by the first key 210A on the second image 214B. In one embodiment, the image enhancement module 206 changes the identified pixels in the second image 214B to black, gray, or other suitable color. In another embodiment, the image enhancement module 206 forms a suitable pattern, such as a stippled pattern, on the identified pixels in the second image 214B. In yet another embodiment, the image enhancement module 206 does not alter the pixels in the second image 214B but rather reduces the brightness of the identified pixels. The image enhancement module 206 may also apply hysteresis on the new second image 214B to prevent flickering at edges of modified pixels, thereby providing a smooth transition between the modified pixels and the surrounding image. The Examiner interprets the projected image on the presenter which is captured in the image would contain distortion caused by the projection on the presenter. It would have been obvious for a person with ordinary skill in the art before the invention was effectively filed to have modified Kurokawa in view of Gruen to have included the features of "[distorted] by an object in front of the projection surface" because the occlusion causes at least a portion of the image to be displayed on the presenter's face and body, which can be embarrassing to the presenter as well as distracting to the audience. Further, the bright light from the projector may blind, irritate, and/or distract the presenter (Gruen [0002]).
In regards to claim 6, Kurokawa/Gruen teach all the limitations of claim 1 and further teach:
"wherein the changing the color or brightness comprises: among a plurality of pixels corresponding to a plurality of values, changing a value of a pixel corresponding to the distorted pattern region based on the value being within a predetermined range"
Gruen paragraph [0022] teaches the image enhancement module 206 may reduce the brightness of the portion of the second image 214B that would have blinded the presenter 114. The Examiner interprets the predetermined range is the range of brightness that would have blinded the presenter. It would have been obvious for a person with ordinary skill in the art before the invention was effectively filed to have modified Kurokawa in view of Gruen to have included the features of "wherein the changing the color or brightness comprises: among a plurality of pixels corresponding to a plurality of values, changing a value of a pixel corresponding to the distorted pattern region based on the value being within a predetermined range" because the occlusion causes at least a portion of the image to be displayed on the presenter's face and body, which can be embarrassing to the presenter as well as distracting to the audience. Further, the bright light from the projector may blind, irritate, and/or distract the presenter (Gruen [0002]).
In regards to claim 8, Kurokawa/Gruen teach all the limitations of claim 1 and further teach:
“wherein the first pattern includes a plurality of patterns among: a plurality of horizontal lines, a plurality of vertical lines, a plurality of diagonal lines, or a plurality of lines at different angles, and2 wherein the adding the first pattern to the first image comprises: adding the plurality of patterns to the first image in sequence”
Kurokawa paragraph [0064] and Figure 5 teaches he projector 101 adds a predetermined structured light pattern to a certain frame of an input image to generate a frame image in which a positive image of the structured light pattern is combined with the input image, and subtracts the structured light pattern from a subsequent frame of the input image to generate a frame image in which a negative image of the structured light pattern is combined with the input image. The Examiner interprets that triangles are a plurality of lines at different angles.
In regards to claim 9, Kurokawa/Gruen teach all the limitations of claim 1 and claim 9 contains similar limitations written is apparatus format. Therefore, claim 9 is rejected for similar reasoning as applied to claim 1.
In regards to claim 14, Kurokawa/Gruen teach all the limitations of claim 9 and claim 14 contains similar limitations as in claim 6. Therefore, claim 14 is rejected for similar reasoning as applied to claim 6.
In regards to claim 15, Kurokawa/Gruen teach all the limitations of claim 1 and claim 15 contains similar limitations written in article format. Therefore, claim 15 is rejected for similar reasoning as applied to claim 1.
In regards to claim 16, Kurokawa teaches:
“A control method of an electronic device, the method comprising: obtaining a source image to be projected”
Kurokawa Figure 5 and paragraph [0047] teaches The image processing unit 202 processes the image to be projected and output from the projection unit 201. The image output from the image processing unit 202 is an input image input from the image input unit 203 and a test pattern held in the image processing unit 102.
“projecting a first pattern onto a projection surface”
Kurokawa paragraph [0038] teaches the projector 101 projects a test pattern on the screen 102 when checking the projection status. In this embodiment, it is assumed that the projection status checking operation is performed by online sensing as described later. Therefore, the projector 101 embeds a test pattern in an original video and projects it on the screen 102.
“acquiring a second image of the projected first pattern”
Kurokawa paragraph [0040] teaches The projector 101 performs geometric correction on the original video based on the correspondence relationship between the original test pattern and the test pattern on the captured image.
“based on the first pattern included in the second image, identifying a pattern region distorted ...”
Kurokawa paragraph [0040] teaches The projector 101 performs geometric correction on the original video based on the correspondence relationship between the original test pattern and the test pattern on the captured image.
“and projecting the source image and the first pattern onto the projection surface”
Kurokawa Figure 15 step 1506 loops back to projecting the source image and the first pattern.
Kurokawa does not explicitly teach:
“[distorted] by an object in front of the projection surface” and “excluding a portion of the source image that corresponds to the distorted pattern region”
Gruen paragraph [0019] teaches The presenter 114 is positioned between the projector 104 and the projection screen 110, thereby occluding a portion of the projection of the first image 214A. As a result, the first projected image 216A contains a shadow 220 of the presenter 114. Gruen paragraph [0019] teaches the captured image 212A contains the projected image displayed on the projection screen 110 as well as any occluding objects (i.e., the presenter 114) between the projector 104 and the projection screen 110. The camera 204 provides the first captured image 212A to the computing device 202. Gruen paragraph [0021] teaches The image enhancement module 206 generates the new second image 218 by modifying the pixels identified by the first key 210A on the second image 214B. In one embodiment, the image enhancement module 206 changes the identified pixels in the second image 214B to black, gray, or other suitable color. In another embodiment, the image enhancement module 206 forms a suitable pattern, such as a stippled pattern, on the identified pixels in the second image 214B. In yet another embodiment, the image enhancement module 206 does not alter the pixels in the second image 214B but rather reduces the brightness of the identified pixels. The image enhancement module 206 may also apply hysteresis on the new second image 214B to prevent flickering at edges of modified pixels, thereby providing a smooth transition between the modified pixels and the surrounding image. The Examiner interprets the projected image on the presenter which is captured in the image would contain distortion caused by the projection on the presenter. It would have been obvious for a person with ordinary skill in the art before the invention was effectively filed to have modified Kurokawa in view of Gruen to have included the features of “[distorted] by an object in front of the projection surface” and “excluding a portion of the source image that corresponds to the distorted pattern region” because the occlusion causes at least a portion of the image to be displayed on the presenter's face and body, which can be embarrassing to the presenter as well as distracting to the audience. Further, the bright light from the projector may blind, irritate, and/or distract the presenter (Gruen [0002]).
In regards to claim 17, Kurokawa/Gruen teach all the limitations of claim 16 and claim 17 contains similar limitations as in claim 1. Therefore, claim 17 is rejected for similar reasoning as applied to claim 1.
Claim(s) 4-5 and 12-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kurokawa in view of Gruen in view of Timoner US 2012/0120372 hereinafter referred to as Timoner.
In regards to claim 4, Kurokawa/Gruen teach all the limitations of claim 1 but do not explicitly teach:
"wherein the projecting the first image onto the projection surface comprises: based on an external projector projecting a third image, different from the first image, onto the projection surface, projecting the first image at a position that at least partially overlaps the third image"
Timoner paragraph [0027] and Figure 1 teaches the outlines 101 and 103 of two projected images from two projectors landing on flat screen region 100. The outlines do not need to be rectangular and are drawn in the Fig. as trapezoids by way of example. A variety of geometric shapes are expressly contemplated in further examples. In overlap region 105, both projectors overlap on the screen 100. It would have been obvious for a person with ordinary skill in the art before the invention was effectively filed to have modified Kurokawa/Gruen in view of Timoner to have included the features or "wherein the projecting the first image onto the projection surface comprises: based on an external projector projecting a third image, different from the first image, onto the projection surface, projecting the first image at a position that at least partially overlaps the third image" because for manual geometric calibration, the state of the art generally involves the application of substantial precision, which is often tedious to achieve (Timoner [0010]).
In regards to claim 5, Kurokawa/Gruen teach all the limitations of claim 4 but do not explicitly teach:
"wherein the first image includes at least one first matching point, and wherein the first image is projected at a position at which the at least one first matching point overlaps at least one second matching point included in the third image"
Timoner paragraph [0027] and Figure 1 teaches in the overlap region 105, four X-O correspondence pairs are shown: X (127) to O (129), X (131) to O (133), X (119) to O (121), and X (123) to O (125). Those correspondence pairs can be used to form a mapping from one projector to another. It would have been obvious for a person with ordinary skill in the art before the invention was effectively filed to have modified Kurokawa/Gruen in viewof Timoner to have included the features or "wherein the first image includes at least one first matching point, and wherein the first image is projected at a position at which the at least one first matching point overlaps at least one second matching point included in the third image" because for manual geometric calibration, the state of the art generally involves the application of substantial precision, which is often tedious to achieve (Timoner [0010]).
In regards to claim 12, Kurokawa/Gruen teach all the limitations of claim 9 and claim 12 contains similar limitations as in claim 4. Therefore, claim 12 is rejected for similar reasoning as applied to claim 4.
In regards to claim 13, Kurokawa/Gruen teach all the limitations of claim 12 and claim 13 contains similar limitations as in claim 5. Therefore, claim 13 is rejected for similar reasoning as applied to claim 5.
Claim(s) 7 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kurokawa in view of Gruen in view of Vieth et al. US 2015/0015699 hereinafter referred to as Vieth.
In regards to claim 7, Kurokawa/Gruen teach all the limitations of claim 1 but do not explicitly teach:
"wherein the source image includes a plurality of frames, and wherein the adding the first pattern to the source image comprises: replacing at least one frame among the plurality of frames with a frame that includes the first pattern"
Vieth teaches in paragraph [0066] then the performance of method 400 is repeated as discussed above, the projection of a canvas image at block 420 and the projection of structured light at block 500 can be substantially simultaneous, or can alternate. For example, when projector 104 is capable of projecting visible and invisible light simultaneously, a canvas image can be projected at the same time as the structured light which will be used to generate the next canvas image. In other examples, the structured light may be projected in between frames (that is, in between projected canvas images), with each frame of structured light being used to generate the subsequent canvas image. It would have been obvious for a person with ordinary skill in the art before the invention was effectively filed to have modified Kurokawa/Gruen in view of Vieth to have included the features or "wherein the source image includes a plurality of frames, and wherein the adding the first pattern to the source image comprises: replacing at least one frame among the plurality of frames with a frame that includes the first pattern" because In addition to the obstacles involved in enabling projection equipment to follow moving objects, the images projected onto those moving objects may still appear distorted. Relative motion between projectors and target objects therefore renders accurate projection difficult to achieve (Vieth [0003]).
In regards to claim 18, Kurokawa/Gruen teach all the limitations of claim 17 and claim 18 contains similar limitations as in claim 7. Therefore, claim 18 is rejected for similar reasoning as applied to claim 7.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL E TEITELBAUM, Ph.D. whose telephone number is (571)270-5996. The examiner can normally be reached 8:30AM-5:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Miller can be reached at 571-272-7353. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL E TEITELBAUM, Ph.D./Primary Examiner, Art Unit 2422
Prosecution Insights