DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement filed 01/29/2024 fails to comply with 37 CFR 1.98(a)(2), which requires a legible copy of each cited foreign patent document; each non-patent literature publication or that portion which caused it to be listed; and all other information or that portion which caused it to be listed. It has been placed in the application file, but the information referred to therein has not been considered. All other IDSs have been fully considered.
Priority
All claims have been examined using the effective fling date of 03/03/2017.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1 and 16, and dependent claims, are rejected under 35 USC 101 as being directed to non-statutory subject matter. A "computer-readable storage medium" is defined in the specification to include " [00114]Although features and elements are described herein in particular combinations, each feature or element can be used alone or in any combination with the other features and elements. For example, the functionality described herein may be described as being performed by a control device, such as a remote control device or a lighting device, but may be similarly performed by a hub device or a network device. The methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted over wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, a read only memory (ROM), a random access memory (RAM), removable disks, and optical media such as CD-ROM disks, and digital versatile disks (DVDs)." (Originally Filed Specification Paragraph [00114]). The broadest reasonable interpretation of a claim drawn to a computer readable medium (also called machine readable medium and other such variations) typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media. See MPEP 2111.01. When the broadest reasonable interpretation of a claim covers a signal per se, the claim must be rejected under 35 U.S.C 101 as covering non-statutory subject matter. The claims, as defined in the specification, cover both non-statutory subject matter and statutory subject matter. A claim drawn to such a computer readable medium that covers both transitory and non-transitory embodiments may be amended to narrow the claim to cover only statutory embodiments by adding the limitation "non-transitory" to the claim.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3 and 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Lundy (US 20150368967 A1) in view of Stam (US 20060177098 A1) and Xiang (CN 102163328 A).
With respect to claim 1, Lundy discloses the least one computer-readable medium having stored thereon instructions (“The system controller 110 may be configured to control the load control devices (e.g., the dimmer switch 120, the LED drivers 130, and/or the motorized roller shades 140)…memory in the system controller 110” paragraph 0040 and “The system controller 1300 may comprise a control circuit 1310, which may include one or more of a processor (e.g., a microprocessor), a microcontroller, a programmable logic device (PLD), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or any suitable processing device. The control circuit 1310 may perform signal coding, data processing, image processing, power control, input/output processing, and/or any other functionality that enables the system controller 1300 to perform” paragraph 0153) that, when executed by at least one control circuit (“The system controller 110 may be configured to control the load control devices (e.g., the dimmer switch 120, the LED drivers 130, and/or the motorized roller shades 140)…memory in the system controller 110” paragraph 0040), are configured to cause the at least one control circuit to: obtain sensor information of a view outside of a window on which a motorized window treatment is installed (“The one or more window sensors 158 may be mounted to the inside surfaces of one or more windows in the space in which the load control system 100 is installed or to the exterior of the building. One or more window sensors 158 may be mounted adjacent to at least one of the motorized window treatments 140. Each window sensor 158 may be battery-powered and/or may be operable to transmit the RF signals 106 to the wireless adapter device 159. The window sensor 158 may receive a sensor reading by measuring an amount of daylight (e.g., daylight intensity level) shining on the window sensor 158. The window sensor may transmit digital messages via the RF signals 106 that include the sensor reading, for example, when the magnitude of the light intensity changes by a predetermined amount (e.g., approximately 20%).” Paragraph 0045) and detect, in response to the luminance measured from the image sensor, the location of the glare condition (“If the total light levels measured by one or more of the window sensors 158 are greater than or equal to the dark-override threshold L.sub.TH-DK, the system controllers 110 may be operable to determine that sunny conditions exist on one or more of the façades 174, and to enable the sunlight penetration limiting mode to control the motorized roller shades 140 to limit the sunlight penetration distance d.sub.PEN 186 in one or more of the spaces 170 (e.g., to prevent sun glare on the table 178 in the space 170)” paragraph 0059);control a shade position of a motorized window treatment based on the location of the glare condition (“As previously mentioned, the system controller 110 may operate in the sunlight penetration limiting mode to control the motorized roller shades 140 to limit the sunlight penetration distance d.sub.PEN 186 to be less than a desired maximum sunlight penetration distance d.sub.MAX. For example, the sunlight penetration distance d.sub.PEN 186 may be limited such that the sunlight does not shine directly on the table 178 to prevent sun glare on the table.” Paragraph 0055)
Lundy does not teach obtain one or more images of a view outside of a window on which a motorized window treatment is installed, each of the one or more images comprising rows of pixels; process the rows of pixels to calculate a respective luminance of multiple pixels of at least one image of the one or more images, wherein the instructions being configured to cause the at least one control circuit to process the rows of pixels comprises the instructions being configured to cause the at least one control circuit to: start at a first pixel on a first row of pixels of the at least one image; and step through each of the multiple pixels on the first row before continuing to a next row of pixels to determine the respective luminance of each of the multiple pixels of the at least one image until a location of a glare condition is detected; cease to process the at least one image upon the detection of the location of the glare condition to avoid determination of the respective luminance for remaining unprocessed pixels of the at least one image; detect, in response to the luminance of at least one of the multiple pixels of the at least one image, the location of the glare condition; and control a shade position of a motorized window treatment based on the location of the glare condition.
Stam teaches an image system able to obtain one or more images of a view outside (“According to another aspect of the invention, a control system is provided to control exterior lights of a vehicle. The control system comprises an image array sensor including a plurality of pixels, an optical system configured to image the scene forward of the controlled vehicle onto the image array sensor” paragraph 0031), each of the one or more images comprising rows of pixels (“The array may have 25 .mu.m or larger photogate active pixels. In particular, the array may include 30 .mu.m or larger pixels arranged in a grid smaller than 200 rows by 200 columns, and may advantageously comprise a rectangular array having 64 columns and 80 rows of pixels” paragraph 0095); process the rows of pixels to calculate a respective luminance of multiple pixels of at least one image of the one or more images (“The output signal from each of the pixels is representative of the illumination sensed by each pixel. This output signal is transferred to the column output circuitry 302 one row at a time.” Paragraph 0141), wherein the instructions being configured to cause the at least one control circuit to process the rows of pixels comprises the instructions being configured to cause the at least one control circuit to: start at a first pixel on a first row of pixels of the at least one image (“The output signal from each of the pixels is representative of the illumination sensed by each pixel. This output signal is transferred to the column output circuitry 302 one row at a time.” Paragraph 0141 and “The next pixel location is determined by raster scanning through the image by first incrementing the X coordinate and examining pixels to the right until the last pixel in the row is reached, and then proceeding to the first pixel in the next row” paragraph 0153); and step through each of the multiple pixels on the first row before continuing to a next row of pixels to determine the respective luminance of each of the multiple pixels (“The output signal from each of the pixels is representative of the illumination sensed by each pixel. This output signal is transferred to the column output circuitry 302 one row at a time.” Paragraph 0141 and “The next pixel location is determined by raster scanning through the image by first incrementing the X coordinate and examining pixels to the right until the last pixel in the row is reached, and then proceeding to the first pixel in the next row” paragraph 0153)
Stam is analogous art in the same field of endeavor as the claimed invention. Stam is directed towards a system for controlling a device based on illumination (“According to another aspect of the invention, a control system is provided to control exterior lights of a vehicle. The control system comprises an image array sensor including a plurality of pixels, an optical system configured to image the scene forward of the controlled vehicle onto the image array sensor” paragraph 0031 and “The output signal from each of the pixels is representative of the illumination sensed by each pixel. This output signal is transferred to the column output circuitry 302 one row at a time.” Paragraph 0141”). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the window treatment control system of Lundy with the luminance based system of Stam by utilizing Stam’s teachings of image pixel luminance based control inside the sensor based system of Lundy, with the expectation that doing so would lead to better control of the luminance of some area (“The perceived brightness and/or glare area of the headlamps (and/or other exterior lights) may be varied by changing the illumination output of one or more lights forming the headlamps, by steering one or more lights to change the aim of one or more of the lights forming the headlamps, activating or deactivating some or all of the lights, altering the illumination pattern forward of the vehicle, or a combination of the above.” Paragraph 0090 and “According to another aspect of the invention, a control system is provided to control exterior lights of a vehicle. The control system comprises an image array sensor including a plurality of pixels, an optical system configured to image the scene forward of the controlled vehicle onto the image array sensor” paragraph 0031 and “The output signal from each of the pixels is representative of the illumination sensed by each pixel. This output signal is transferred to the column output circuitry 302 one row at a time.” Paragraph 0141”).
Xiang teaches processing of the at least one image until a location of a glare condition is detected (see Examiner figure 1); cease to process the at least one image upon the detection of the location of the glare condition to avoid determination of the respective luminance for remaining unprocessed pixels of the at least one image (see Examiner figure 1);
PNG
media_image1.png
587
690
media_image1.png
Greyscale
Examiner figure 1: Translated figure 1 of Xiang
Xiang is analogous art in the same field of endeavor as the claimed invention. Xiang discloses locating glare in images (“The invention claims the detecting a glare array can be generated to highlight in the image to locate and eliminate and elimination of glare in traffic video image method.” Paragraph 0001). A person of ordinary skill, before the effective fling date of the claimed invention, would have found it obvious to combine the system of Lundy and Stam with Xiang by utilizing Xiang’s image glare identification processing strategy in conjunction with the combined system’s glare processing methodology, with the expectation that doing so would lead to better glare location determining (“accurate positioning of the glare and glare elimination effect better” page 4 last paragraph)
With respect to claim 2, Lundy, Stam, and Xiang teach the at least one computer readable medium of claim 1. Lundy further teaches wherein the instructions being configured to cause the at least one control circuit to control the shade position of the motorized window treatment based on the location of the glare condition further comprises the instructions being configured to cause the at least one control circuit to transmit a shade control command (“As previously mentioned, the system controller 110 may operate in the sunlight penetration limiting mode to control the motorized roller shades 140 to limit the sunlight penetration distance d.sub.PEN 186 to be less than a desired maximum sunlight penetration distance d.sub.MAX. For example, the sunlight penetration distance d.sub.PEN 186 may be limited such that the sunlight does not shine directly on the table 178 to prevent sun glare on the table.” Paragraph 0055).
With respect to claim 3, Lundy, Stam, and Xiang teach the at least one computer readable medium of claim 1. Xiang further teaches wherein the instructions, when executed, are further configured to cause the at least one control circuit to determine a background luminance representative of a luminance of a background of the at least one image (“obtaining the background image through background updating method. then using the proposed glare detecting method” paragraph 0010).
With respect to claim 13, Lundy, Stam, and Xiang teach the at least one computer readable medium of claim 1. Xiang further teaches wherein the instructions, when executed, are further configured to cause the at least one control circuit to detect the location of the glare condition when the luminance of a specific pixel is greater than a luminance threshold (“glare detecting method is the pixel average value Obtaining average values of local accumulation gradient image every one row pixel and the whole local accumulation gradient image; then, through more than two average ratio and setting of the second threshold comparing. detecting the glare” page 2 bottom page 3 top).
With respect to claim 14, Lundy, Stam, and Xiang teach the at least one computer readable medium of claim 1. Stam further teaches wherein the first row is a bottom row of the at least one image (“The output signal from each of the pixels is representative of the illumination sensed by each pixel. This output signal is transferred to the column output circuitry 302 one row at a time.” Paragraph 0141 and “The next pixel location is determined by raster scanning through the image by first incrementing the X coordinate and examining pixels to the right until the last pixel in the row is reached, and then proceeding to the first pixel in the next row” paragraph 0153), and wherein the next row of pixels is immediately above the bottom row of the at least one image (“The output signal from each of the pixels is representative of the illumination sensed by each pixel. This output signal is transferred to the column output circuitry 302 one row at a time.” Paragraph 0141 and “The next pixel location is determined by raster scanning through the image by first incrementing the X coordinate and examining pixels to the right until the last pixel in the row is reached, and then proceeding to the first pixel in the next row” paragraph 0153).
With respect to claim 15, Lundy, Stam, and Xiang teach the at least one computer readable medium of claim 1. Stam further teaches wherein the instructions being configured to cause the at least one control circuit to process the rows of pixels further comprises the instructions being configured to cause the at least one control circuit to calculate the respective luminance of each of the multiple pixels (“The output signal from each of the pixels is representative of the illumination sensed by each pixel. This output signal is transferred to the column output circuitry 302 one row at a time.” Paragraph 0141 and “The next pixel location is determined by raster scanning through the image by first incrementing the X coordinate and examining pixels to the right until the last pixel in the row is reached, and then proceeding to the first pixel in the next row” paragraph 0153).
Claims 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Lundy, Stam, and Xiang as applied to claim 1 above, and further in view of Johnson (US 20130332866 A1).
With respect to claim 4, Lundy, Stam, and Xiang teach the at least one computer readable medium of claim 1, but do not teach the rest of the limitations. Johnson further teaches reordering the pixels of the at least one image from a darkest pixel to a lightest pixel (“In other embodiments, the overall intensity of the image is based on the brightness value of the median pixel in the distribution of pixels of the histogram. The median pixel in the distribution of pixels of the histogram is the median pixel of all the pixels of the image sequentially ordered.” Paragraph 0047); and calculating the luminance of a pixel that is a predetermined percentage of a distance from the darkest pixel to the lightest pixel (“The distance to black 750, in some embodiments, is a measurement from the black edge of the histogram (e.g., the darkest possible tonal value of the image) to a particular percentile of the histogram values. In other words, a percentage of the total number of image pixels is specified which encompasses the darkest pixels within the specified percentile” Paragraph 0101).
Johnson is analogous art to the claimed invention pertinent to solving the problem of pixel luminance processing. Johnson is directed toward a luminance histogram based method of exposure adjustment (“In some embodiments, the method computes a histogram based on a set of luminance values of the image and identifies a distance to a particular point in the histogram in order to identify the multiplier value for adjusting the luminance of the image.” Paragraph 0004). A person of ordinary skill in the art before the effective filing date of the claimed invention, would have found it obvious to combine the system of Lundy, Stam, and Xiang with Johnson by utilizing Johnson’s luminance ordering strategy to more efficiently process pixel luminance with the expectation that doing so would lead to access to important information such as relative pixel light intensity (“In other embodiments, the overall intensity of the image is based on the brightness value of the median pixel in the distribution of pixels of the histogram. The median pixel in the distribution of pixels of the histogram is the median pixel of all the pixels of the image sequentially ordered.” Paragraph 0047).
With respect to claim 5, Lundy, Stam, Xiang and Johnson teach the at least one computer readable medium of claim 4. Johnson further teaches wherein the predetermined percentage is 25% (“While this example describes the specified percentage as 1% of the total number of pixels for the image, other percentages (e.g., 10%, 0.1%, etc.) can be used in other embodiments.” Paragraph 0101 and figure 7).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Lundy, Stam, and Xiang as applied to claim 3 above, and further in view of Ackermann (WO 2010116274 A1).
With respect to claim 6, Lundy, Stam, and Xiang teach the at least one computer readable medium of claim 3. Ackermann further teaches detecting the location of the glare condition when a ratio of the luminance of a specific pixel to the background luminance is greater than a predetermined contrast threshold (“The control unit 5 then determines the luminance contrast ratio of stripes of the stripe pattern, i.e. by comparing the stripe having the highest luminance with the stripe having the lowest luminance. A control of the blind 3 is conducted in case the contrast ratio is higher than 10:2, to reduce glare” page 12 lines 19-22).
Ackermann is analogous art in the same field of endeavor as the claimed invention. Ackermann is directed towards a system for controlling a shade device (“The invention relates to a system and a method for controlling a shading device with a plurality of controllable shading elements.” Page 1 Field of the Invention). A person of ordinary skill before the effective filing date of the claimed invention would have found it obvious to combine the system of Ackermann with Lundy, Stam, and Xiang, by utilizing Ackermann’s teaching of contrast ratio’s relationship to glare within the combined system’s glare identification process, with the expectation that doing so would lead to a more adaptable system able to recognize glare and provide comfort to users (“The inventive system thus allows to reduce or preferably eliminate direct sunlight transmission, so that thermal discomfort and glare resulting thereof can be advantageously reduced. “ page 2 lines 20-23).
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Lundy, Stam, and Xiang as applied to claim 1 above, and further in view of Berman (CA 2734846 A1).
With respect to claim 7, Lundy, Stam, and Xiang teach the at least one computer readable medium of claim 1. Xiang further teaches wherein the instructions, when executed, are further configured to cause the at least one control circuit to detect the location of the glare condition in response to the luminance of a specific pixel (“glare detecting method is the pixel average value Obtaining average values of local accumulation gradient image every one row pixel and the whole local accumulation gradient image; then, through more than two average ratio and setting of the second threshold comparing. detecting the glare” page 2 bottom – page 3 top), but does not teach wherein the instructions, when executed are further configured to cause the at least one control circuit to: calculate a profile angle of the glare source in response to the location of the specific pixel in the at least one image; and determine the shade position to which to control the motorized window treatment in response to the profile angle.
Berman teaches calculating a profile angle of the glare source in response to the location of the specific pixel in the at least one image (see figure 6); and determine the shade position to which to control the motorized window treatment in response to the profile angle (see figure 6).
Berman is analogous art in the same field of endeavor as the claimed invention. Berman is directed towards “automatic shade control” ( page 1 field of invention). A person of ordinary skill, before the effective filing date of the claimed invention would have found it obvious to combine the system of Lundy, Stam, and Xiang with Berman by utilizing Berman’s shade control in response to profiling angle inside the combined system’s shade control scheme, with the expectation that doing so would lead to a control scheme that can preserve energy efficiency (“Thus, a need exists to manage and control the amount of solar load, solar penetration, and temperatures of the ,window wall. In addition, there is a need to control the amount of solar radiation and brightness to acceptable norms that protect the comfort and health of the occupants, e.g. an energy conserving integrated sub-system.” Page 2 lines 4-7 and “A window covering is adjusted responsive to the calculated reflected light at the location of interest.” Page 2 lines 13-14 ).
With respect to claim 8, Lundy, Stam, Xiang and Berman teach the at least one computer readable medium of claim 7. Berman further teaches wherein the instructions being configured to cause the at least one control circuit to control the shade position of the motorized window treatment based on the location of the glare condition further comprises the instructions being configured to cause the at least one control circuit to transmit a shade control command in response to the shade position determined by the profile angle (see figure 6).
Claims 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over Lundy, Stam, and Xiang as applied to claim 1 above, and further in view of Andrus (US 20100195901 A1).
With respect to claim 9, Lundy, Stam, and Xiang teach the at least one computer readable medium of claim 1, but do not teach wherein the at least one image comprises a high dynamic-range (HDR) image.
Andrus teaches analyzing pixel luminance in an HDR image (“FIG. 12 depicts a system shown in block diagram form that includes one or more imaging devices which preferably have high dynamic range capabilities paragraph 0105” and “Pixels of an image selected and assembled for tone mapping and display are transmitted over path 1219 to Image pixel Luminance Extraction unit 1210” paragraph 0105).
Andrus is analogous art to the claimed invention, pertinent to solving the problem of pixel luminance analysis in HDR or warped images. Andrus is directed towards image processing techniques (“Pixels of an image selected and assembled for tone mapping and display are transmitted over path 1219 to Image pixel Luminance Extraction unit 1210” paragraph 0105). A person of ordinary skill in the art before the effective filing date of the claimed invention, would have found it obvious to combine the system of Lundy, Stam, and Xiang with Andrus by utilizing Andrus’s image processing techniques inside the combined system’s pixel luminance methodology, with the expectation that doing so would result in a more adaptable system able to handle different types of images (FIG. 12 depicts a system shown in block diagram form that includes one or more imaging devices which preferably have high dynamic range capabilities paragraph 0105” and “The Selection and Editing unit 1209 is preferably capable of image selection, of adding text and optional graphics to image data, of optional image processing such as Bayer color interpolation, grouping of images, de-warping and resizing of images, image stitching, and optional feature recognition within images.” Paragraph 0105).
With respect to claim 10, Lundy, Stam, Xiang and Andrus teach the at least one computer readable medium of claim 9. Andrus further teaches retrieving the respective luminance of each of the multiple pixels from data of the HDR image (“FIG. 12 depicts a system shown in block diagram form that includes one or more imaging devices which preferably have high dynamic range capabilities paragraph 0105” and “Pixels of an image selected and assembled for tone mapping and display are transmitted over path 1219 to Image pixel Luminance Extraction unit 1210” paragraph 0105).
With respect to claim 11, Lundy, Stam, and Xiang teach the at least one computer readable medium of claim 1, but do not teach any further limitations. Andrus further teaches wherein the at least one image comprises a warped image (“The Selection and Editing unit 1209 is preferably capable of image selection, of adding text and optional graphics to image data, of optional image processing such as Bayer color interpolation, grouping of images, de-warping and resizing of images, image stitching, and optional feature recognition within images.” Paragraph 0105).
Andrus is analogous art to the claimed invention, pertinent to solving the problem of pixel luminance analysis in HDR or warped images. Andrus is directed towards image processing techniques (“Pixels of an image selected and assembled for tone mapping and display are transmitted over path 1219 to Image pixel Luminance Extraction unit 1210” paragraph 0105). A person of ordinary skill in the art before the effective filing date of the claimed invention, would have found it obvious to combine the system of Lundy, Stam, and Xiang with Andrus by utilizing Andrus’s image processing techniques inside the combined system’s pixel luminance methodology, with the expectation that doing so would result in a more adaptable system able to handle different types of images (FIG. 12 depicts a system shown in block diagram form that includes one or more imaging devices which preferably have high dynamic range capabilities paragraph 0105” and “The Selection and Editing unit 1209 is preferably capable of image selection, of adding text and optional graphics to image data, of optional image processing such as Bayer color interpolation, grouping of images, de-warping and resizing of images, image stitching, and optional feature recognition within images.” Paragraph 0105).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Lundy, Stam, and Xiang as applied to claim 1 above, and further in view of Andrus and Liu (CN 105513071 A).
With respect to claim 12, Lundy, Stam, and Xiang teach the at least one computer readable medium of claim 1, but do not teach further limitations. Andrus teaches wherein the one or more images comprises a warped image, and wherein the instructions, when executed, are further configured to cause the at least one control circuit to generate a non-warped image of the warped image (“The Selection and Editing unit 1209 is preferably capable of image selection, of adding text and optional graphics to image data, of optional image processing such as Bayer color interpolation, grouping of images, de-warping and resizing of images, image stitching, and optional feature recognition within images.” Paragraph 0105).
Andrus is analogous art to the claimed invention, pertinent to solving the problem of pixel luminance analysis in HDR or warped images. Andrus is directed towards image processing techniques (“Pixels of an image selected and assembled for tone mapping and display are transmitted over path 1219 to Image pixel Luminance Extraction unit 1210” paragraph 0105). A person of ordinary skill in the art before the effective filing date of the claimed invention, would have found it obvious to combine the system of Lundy, Stam, and Xiang with Andrus by utilizing Andrus’s image processing techniques inside the combined system’s pixel luminance methodology, with the expectation that doing so would result in a more adaptable system able to handle different types of images (FIG. 12 depicts a system shown in block diagram form that includes one or more imaging devices which preferably have high dynamic range capabilities paragraph 0105” and “The Selection and Editing unit 1209 is preferably capable of image selection, of adding text and optional graphics to image data, of optional image processing such as Bayer color interpolation, grouping of images, de-warping and resizing of images, image stitching, and optional feature recognition within images.” Paragraph 0105).
Liu further teaches de-warping by generating rows of constant profile angle from the warped image (“The space restriction detecting Harris angular point forming topographic pattern effective matching of the profile angle point and image correction” page 6 paragraph 1 and “according to the size of the image into N multiplying M divides the data into a size of data block of L * L, wherein N is column number of pixels, M is the row number of pixels, L unit is pixel, to minimum image block comprises intersection of inner profile line intersecting at the four corners of the image determines the minimum of L is image block, subsequent matching effect dynamically determining the L value, until the spatial relationship of L = N or L = M and blocking data selecting the original four corners of four image data as detection data” page 6 paragraph 10 and “to obtain the matching angular point respectively on the reference image and the image to be corrected for spatial constraint;” page 6 paragraph 14 and “graph-profile angle in point (g) The formation of effective matching to correct image data.” Page 7 paragraph 1), and wherein the instructions being configured to cause the at least one control circuit to process the rows of pixels further comprises the instructions being configured to cause the at least one control circuit to process the non-warped image (“The space restriction detecting Harris angular point forming topographic pattern effective matching of the profile angle point and image correction” page 6 paragraph 1 and “according to the size of the image into N multiplying M divides the data into a size of data block of L * L, wherein N is column number of pixels, M is the row number of pixels, L unit is pixel, to minimum image block comprises intersection of inner profile line intersecting at the four corners of the image determines the minimum of L is image block, subsequent matching effect dynamically determining the L value, until the spatial relationship of L = N or L = M and blocking data selecting the original four corners of four image data as detection data” page 6 paragraph 10 and “to obtain the matching angular point respectively on the reference image and the image to be corrected for spatial constraint;” page 6 paragraph 14 and “graph-profile angle in point (g) The formation of effective matching to correct image data.” Page 7 paragraph 1).
Liu is analogous art to the claimed invention pertinent to solving the problem of warped image processing. Liu is directed towards image processing techniques (“The space restriction detecting Harris angular point forming topographic pattern effective matching of the profile angle point and image correction” page 6 paragraph 1). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the system of Lundy, Stam, Xiang, and Andrus with Liu by utilizing Liu’s de-warping process in lieu of Andrus’, with the expectation that doing so would result in better graphic quality (“The invention belongs to the technical field of graphic quality evaluation, specifically claims a quality evaluation method for map format” page 1 technology field).
Claims 16-24 are rejected under 35 U.S.C. 103 as being unpatentable over Lundy in view of Xiang, Johnson and Ackermann.
With respect to claim 16, Lundy teaches at least one computer-readable medium having stored thereon instructions (“The system controller 110 may be configured to control the load control devices (e.g., the dimmer switch 120, the LED drivers 130, and/or the motorized roller shades 140)…memory in the system controller 110” paragraph 0040 and “The system controller 1300 may comprise a control circuit 1310, which may include one or more of a processor (e.g., a microprocessor), a microcontroller, a programmable logic device (PLD), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or any suitable processing device. The control circuit 1310 may perform signal coding, data processing, image processing, power control, input/output processing, and/or any other functionality that enables the system controller 1300 to perform” paragraph 0153) that, when executed by at least one control circuit (“The system controller 110 may be configured to control the load control devices (e.g., the dimmer switch 120, the LED drivers 130, and/or the motorized roller shades 140)…memory in the system controller 110” paragraph 0040) cause the at least one control circuit to: retrieve one or more images (“The control circuit 1310 may perform signal coding, data processing, image processing, power control, input/output processing, and/or any other functionality that enables the system controller 1300 to perform as described herein.” Paragraph 153) and control a shade position of a motorized window treatment based on the location of the glare condition (“As previously mentioned, the system controller 110 may operate in the sunlight penetration limiting mode to control the motorized roller shades 140 to limit the sunlight penetration distance d.sub.PEN 186 to be less than a desired maximum sunlight penetration distance d.sub.MAX. For example, the sunlight penetration distance d.sub.PEN 186 may be limited such that the sunlight does not shine directly on the table 178 to prevent sun glare on the table.” Paragraph 0055)
Lundy does not teach calculating a background luminance of an image of the one or more images, wherein the background luminance is a value from which a location of a glare condition is detected for controlling a shade position of a motorized window treatment; reorder pixels of the image from darkest to lightest; calculate a luminance of a first pixel that is a predetermined percentage of a distance from the darkest pixel to the lightest pixel; set the luminance of the first pixel as the background luminance; and detect the location of the glare condition when a ratio of the luminance of a second pixel to the background luminance is greater than a predetermined contrast threshold.
Xiang teaches calculate a background luminance of an image of the one or more images (“obtaining the background image through background updating method. then using the proposed glare detecting method” paragraph 0010) and setting a pixel value as the background luminance (“pixel average value” page 2 bottom page 3 top), wherein the background luminance is a value from which a location of a glare condition is detected (“glare detecting method is the pixel average value Obtaining average values of local accumulation gradient image every one row pixel and the whole local accumulation gradient image; then, through more than two average ratio and setting of the second threshold comparing. detecting the glare” page 2 bottom page 3 top)
Xiang is analogous art in the same field of endeavor as the claimed invention. Xiang discloses locating glare in images (“The invention claims the detecting a glare array can be generated to highlight in the image to locate and eliminate and elimination of glare in traffic video image method.” Paragraph 0001). A person of ordinary skill, before the effective fling date of the claimed invention, would have found it obvious to combine the system of Lundy with Xiang by utilizing Xiang’s image glare identification processing strategy in conjunction with Lundy’s luminance processing methodology, with the expectation that doing so would lead to better glare location determining (“accurate positioning of the glare and glare elimination effect better” page 4 last paragraph)
Johnson teaches reordering pixels of the image from darkest to lightest (“In other embodiments, the overall intensity of the image is based on the brightness value of the median pixel in the distribution of pixels of the histogram. The median pixel in the distribution of pixels of the histogram is the median pixel of all the pixels of the image sequentially ordered.” Paragraph 0047); calculating a luminance of a first pixel that is a predetermined percentage of a distance from the darkest pixel to the lightest pixel (“The distance to black 750, in some embodiments, is a measurement from the black edge of the histogram (e.g., the darkest possible tonal value of the image) to a particular percentile of the histogram values. In other words, a percentage of the total number of image pixels is specified which encompasses the darkest pixels within the specified percentile” Paragraph 0101);
Johnson is analogous art to the claimed invention pertinent to solving the problem of pixel luminance processing. Johnson is directed toward a luminance histogram based method of exposure adjustment (“In some embodiments, the method computes a histogram based on a set of luminance values of the image and identifies a distance to a particular point in the histogram in order to identify the multiplier value for adjusting the luminance of the image.” Paragraph 0004). A person of ordinary skill in the art before the effective filing date of the claimed invention, would have found it obvious to combine the system of Lundy, and Xiang with Johnson by utilizing Johnson’s luminance ordering strategy to more efficiently process pixel luminance with the expectation that doing so would lead to access to important information such as relative pixel light intensity (“In other embodiments, the overall intensity of the image is based on the brightness value of the median pixel in the distribution of pixels of the histogram. The median pixel in the distribution of pixels of the histogram is the median pixel of all the pixels of the image sequentially ordered.” Paragraph 0047).
Ackermann teaches detecting the location of the glare condition when a ratio of the luminance of a second pixel to the background luminance is greater than a predetermined contrast threshold (“The control unit 5 then determines the luminance contrast ratio of stripes of the stripe pattern, i.e. by comparing the stripe having the highest luminance with the stripe having the lowest luminance. A control of the blind 3 is conducted in case the contrast ratio is higher than 10:2, to reduce glare” page 12 lines 19-22).
Ackermann is analogous art in the same field of endeavor as the claimed invention. Ackermann is directed towards a system for controlling a shade device (“The invention relates to a system and a method for controlling a shading device with a plurality of controllable shading elements.” Page 1 Field of the Invention). A person of ordinary skill before the effective filing date of the claimed invention would have found it obvious to combine the system of Ackermann with Lundy, Xiang and Johnson, by utilizing Ackermann’s teaching of contrast ratio’s relationship to glare within the combined system’s glare identification process, with the expectation that doing so would lead to a more adaptable system able to recognize glare and provide comfort to users (“The inventive system thus allows to reduce or preferably eliminate direct sunlight transmission, so that thermal discomfort and glare resulting thereof can be advantageously reduced. “ page 2 lines 20-23).
With respect to claim 17, Lundy, Xiang, Johnson and Ackermann teach the at least one computer readable medium of claim 16. Johnson further teaches wherein the predetermined percentage is 25% (“While this example describes the specified percentage as 1% of the total number of pixels for the image, other percentages (e.g., 10%, 0.1%, etc.) can be used in other embodiments.” Paragraph 0101 and figure 7).
With respect to claim 18, Lundy, Xiang, Johnson and Ackermann teach the at least one computer readable medium of claim 16. Xiang further teaches wherein the instructions, when executed, are further configured to cause the at least one control circuit to cease processing the one or more images upon detection of the location of the glare condition (see Examiner figure 1).
With respect to claim 19, Lundy teaches a method of detecting a location of a glare condition from which a shade position of a motorized window treatment is controlled control a shade position of a motorized window treatment based on the location of the glare condition (“As previously mentioned, the system controller 110 may operate in the sunlight penetration limiting mode to control the motorized roller shades 140 to limit the sunlight penetration distance d.sub.PEN 186 to be less than a desired maximum sunlight penetration distance d.sub.MAX. For example, the sunlight penetration distance d.sub.PEN 186 may be limited such that the sunlight does not shine directly on the table 178 to prevent sun glare on the table.” Paragraph 0055)
Lundy does not teach recording one or more images; calculating a background luminance of an image of the one or more images, wherein the background luminance is a value from which the location of the glare condition is detected for controlling the shade position of the motorized window treatment; reordering pixels of the image from darkest to lightest; calculating a luminance of a first pixel that is a predetermined percentage of a distance from the darkest pixel to the lightest pixel; setting the luminance of the first pixel as the background luminance; and detecting the location of the glare condition when a ratio of the luminance of a second pixel to the background luminance is greater than a predetermined contrast threshold.
Xiang teaches recording one or more images (“The invention claims a detection for traffic video image in glare and eliminating method” page 2 paragraph 0006 and examiner figure 1) calculate a background luminance of an image of the one or more images (“obtaining the background image through background updating method. then using the proposed glare detecting method” paragraph 0010) and setting a pixel value as the background luminance (“pixel average value” page 2 bottom page 3 top), wherein the background luminance is a value from which a location of a glare condition is detected (“glare detecting method is the pixel average value Obtaining average values of local accumulation gradient image every one row pixel and the whole local accumulation gradient image; then, through more than two average ratio and setting of the second threshold comparing. detecting the glare” page 2 bottom page 3 top)
Xiang is analogous art in the same field of endeavor as the claimed invention. Xiang discloses locating glare in images (“The invention claims the detecting a glare array can be generated to highlight in the image to locate and eliminate and elimination of glare in traffic video image method.” Paragraph 0001). A person of ordinary skill, before the effective fling date of the claimed invention, would have found it obvious to combine the system of Lundy with Xiang by utilizing Xiang’s image glare identification processing strategy in conjunction with Lundy’s luminance processing methodology, with the expectation that doing so would lead to better glare location determining (“accurate positioning of the glare and glare elimination effect better” page 4 last paragraph)
Johnson teaches reordering pixels of the image from darkest to lightest (“In other embodiments, the overall intensity of the image is based on the brightness value of the median pixel in the distribution of pixels of the histogram. The median pixel in the distribution of pixels of the histogram is the median pixel of all the pixels of the image sequentially ordered.” Paragraph 0047); calculating a luminance of a first pixel that is a predetermined percentage of a distance from the darkest pixel to the lightest pixel (“The distance to black 750, in some embodiments, is a measurement from the black edge of the histogram (e.g., the darkest possible tonal value of the image) to a particular percentile of the histogram values. In other words, a percentage of the total number of image pixels is specified which encompasses the darkest pixels within the specified percentile” Paragraph 0101);
Johnson is analogous art to the claimed invention pertinent to solving the problem of pixel luminance processing. Johnson is directed toward a luminance histogram based method of exposure adjustment (“In some embodiments, the method computes a histogram based on a set of luminance values of the image and identifies a distance to a particular point in the histogram in order to identify the multiplier value for adjusting the luminance of the image.” Paragraph 0004). A person of ordinary skill in the art before the effective filing date of the claimed invention, would have found it obvious to combine the system of Lundy, and Xiang with Johnson by utilizing Johnson’s luminance ordering strategy to more efficiently process pixel luminance with the expectation that doing so would lead to access to important information such as relative pixel light intensity (“In other embodiments, the overall intensity of the image is based on the brightness value of the median pixel in the distribution of pixels of the histogram. The median pixel in the distribution of pixels of the histogram is the median pixel of all the pixels of the image sequentially ordered.” Paragraph 0047).
Ackermann teaches detecting the location of the glare condition when a ratio of the luminance of a second pixel to the background luminance is greater than a predetermined contrast threshold (“The control unit 5 then determines the luminance contrast ratio of stripes of the stripe pattern, i.e. by comparing the stripe having the highest luminance with the stripe having the lowest luminance. A control of the blind 3 is conducted in case the contrast ratio is higher than 10:2, to reduce glare” page 12 lines 19-22).
Ackermann is analogous art in the same field of endeavor as the claimed invention. Ackermann is directed towards a system for controlling a shade device (“The invention relates to a system and a method for controlling a shading device with a plurality of controllable shading elements.” Page 1 Field of the Invention). A person of ordinary skill before the effective filing date of the claimed invention would have found it obvious to combine the system of Ackermann with Lundy, Xiang and Johnson, by utilizing Ackermann’s teaching of contrast ratio’s relationship to glare within the combined system’s glare identification process, with the expectation that doing so would lead to a more adaptable system able to recognize glare and provide comfort to users (“The inventive system thus allows to reduce or preferably eliminate direct sunlight transmission, so that thermal discomfort and glare resulting thereof can be advantageously reduced. “ page 2 lines 20-23).
With respect to claim 20, Lundy, Xiang, Johnson and Ackermann teach the method of claim 19. Johnson further teaches wherein the predetermined percentage is 25% (“While this example describes the specified percentage as 1% of the total number of pixels for the image, other percentages (e.g., 10%, 0.1%, etc.) can be used in other embodiments.” Paragraph 0101 and figure 7).
With respect to claim 21, Lundy, Xiang, Johnson and Ackermann teach the method of claim 19. Xiang further teaches ceasing to process the one or more images upon detection of the location of the glare condition (see Examiner figure 1).
With respect to claim 22, Lundy teaches a device for detecting a location of a glare condition from which a shade position of a motorized window treatment is controlled (“As previously mentioned, the system controller 110 may operate in the sunlight penetration limiting mode to control the motorized roller shades 140 to limit the sunlight penetration distance d.sub.PEN 186 to be less than a desired maximum sunlight penetration distance d.sub.MAX. For example, the sunlight penetration distance d.sub.PEN 186 may be limited such that the sunlight does not shine directly on the table 178 to prevent sun glare on the table.” Paragraph 0055), the device comprising: a memory (“The system controller 110 may be configured to control the load control devices (e.g., the dimmer switch 120, the LED drivers 130, and/or the motorized roller shades 140)…memory in the system controller 110” paragraph 0040 and “The system controller 1300 may comprise a control circuit 1310, which may include one or more of a processor (e.g., a microprocessor), a microcontroller, a programmable logic device (PLD), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or any suitable processing device. The control circuit 1310 may perform signal coding, data processing, image processing, power control, input/output processing, and/or any other functionality that enables the system controller 1300 to perform” paragraph 0153) and controlling a shade position of a motorized window treatment based on the location of the glare condition (“As previously mentioned, the system controller 110 may operate in the sunlight penetration limiting mode to control the motorized roller shades 140 to limit the sunlight penetration distance d.sub.PEN 186 to be less than a desired maximum sunlight penetration distance d.sub.MAX. For example, the sunlight penetration distance d.sub.PEN 186 may be limited such that the sunlight does not shine directly on the table 178 to prevent sun glare on the table.” Paragraph 0055)
Xiang teaches calculate a background luminance of an image of the one or more images (“obtaining the background image through background updating method. then using the proposed glare detecting method” paragraph 0010) and setting a pixel value as the background luminance (“pixel average value” page 2 bottom page 3 top), wherein the background luminance is a value from which a location of a glare condition is detected (“glare detecting method is the pixel average value Obtaining average values of local accumulation gradient image every one row pixel and the whole local accumulation gradient image; then, through more than two average ratio and setting of the second threshold comparing. detecting the glare” page 2 bottom page 3 top)
Xiang is analogous art in the same field of endeavor as the claimed invention. Xiang discloses locating glare in images (“The invention claims the detecting a glare array can be generated to highlight in the image to locate and eliminate and elimination of glare in traffic video image method.” Paragraph 0001). A person of ordinary skill, before the effective fling date of the claimed invention, would have found it obvious to combine the system of Lundy with Xiang by utilizing Xiang’s image glare identification processing strategy in conjunction with Lundy’s luminance processing methodology, with the expectation that doing so would lead to better glare location determining (“accurate positioning of the glare and glare elimination effect better” page 4 last paragraph)
Johnson teaches a system configured to store one or more images (see figure 4); retrieving an image of the one or more images from memory (see figure 4) and reordering pixels of the image from darkest to lightest (“In other embodiments, the overall intensity of the image is based on the brightness value of the median pixel in the distribution of pixels of the histogram. The median pixel in the distribution of pixels of the histogram is the median pixel of all the pixels of the image sequentially ordered.” Paragraph 0047); calculating a luminance of a first pixel that is a predetermined percentage of a distance from the darkest pixel to the lightest pixel (“The distance to black 750, in some embodiments, is a measurement from the black edge of the histogram (e.g., the darkest possible tonal value of the image) to a particular percentile of the histogram values. In other words, a percentage of the total number of image pixels is specified which encompasses the darkest pixels within the specified percentile” Paragraph 0101);
Johnson is analogous art to the claimed invention pertinent to solving the problem of pixel luminance processing. Johnson is directed toward a luminance histogram based method of exposure adjustment (“In some embodiments, the method computes a histogram based on a set of luminance values of the image and identifies a distance to a particular point in the histogram in order to identify the multiplier value for adjusting the luminance of the image.” Paragraph 0004). A person of ordinary skill in the art before the effective filing date of the claimed invention, would have found it obvious to combine the system of Lundy, and Xiang with Johnson by utilizing Johnson’s luminance ordering strategy to more efficiently process pixel luminance with the expectation that doing so would lead to access to important information such as relative pixel light intensity (“In other embodiments, the overall intensity of the image is based on the brightness value of the median pixel in the distribution of pixels of the histogram. The median pixel in the distribution of pixels of the histogram is the median pixel of all the pixels of the image sequentially ordered.” Paragraph 0047).
Ackermann teaches detecting the location of the glare condition when a ratio of the luminance of a second pixel to the background luminance is greater than a predetermined contrast threshold (“The control unit 5 then determines the luminance contrast ratio of stripes of the stripe pattern, i.e. by comparing the stripe having the highest luminance with the stripe having the lowest luminance. A control of the blind 3 is conducted in case the contrast ratio is higher than 10:2, to reduce glare” page 12 lines 19-22).
Ackermann is analogous art in the same field of endeavor as the claimed invention. Ackermann is directed towards a system for controlling a shade device (“The invention relates to a system and a method for controlling a shading device with a plurality of controllable shading elements.” Page 1 Field of the Invention). A person of ordinary skill before the effective filing date of the claimed invention would have found it obvious to combine the system of Ackermann with Lundy, Xiang and Johnson, by utilizing Ackermann’s teaching of contrast ratio’s relationship to glare within the combined system’s glare identification process, with the expectation that doing so would lead to a more adaptable system able to recognize glare and provide comfort to users (“The inventive system thus allows to reduce or preferably eliminate direct sunlight transmission, so that thermal discomfort and glare resulting thereof can be advantageously reduced. “ page 2 lines 20-23).
With respect to claim 23, Lundy, Xiang, Johnson and Ackermann teach the device of claim 22. Johnson further teaches wherein the predetermined percentage is 25% (“While this example describes the specified percentage as 1% of the total number of pixels for the image, other percentages (e.g., 10%, 0.1%, etc.) can be used in other embodiments.” Paragraph 0101 and figure 7).
With respect to claim 24, Lundy, Xiang, Johnson and Ackermann teach the device of claim 22. Xiang further teaches wherein the control circuit is further configured to cease processing the one or more images upon detection of the location of the glare condition (see Examiner figure 1).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Colson (US 20140090787 A1): discloses motorized blind control system
Kim (US 20130088463 A1): discloses illumination-based image processing
Hall (US 20150285535 A1): discloses climate based automatic blind control
Any inquiry concerning this communication or earlier communications from the examiner should be directed to REBECCA C WILLIAMS whose telephone number is (571)272-7074. The examiner can normally be reached M-F 7:30am - 4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew W Bee can be reached at (571)270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/REBECCA COLETTE WILLIAMS/Examiner, Art Unit 2677
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666